Create llms.txt for your website (WordLift create-llms-txt) API

Create a useful, structured index of your website. It discovers important public pages within robots.txt and hands you a ready llms.txt file, plus a draft

Input

Send these Fields as input. A Field not listed here is refused.

website_url

Your website URL

  • Type: string
  • Required: yes
  • Length in characters: 1 to 2000

pages

Pages to inspect

  • Type: number
  • Required: no
  • Value: 2 to 30

discovery_mode

Discovery mode

  • Type: string
  • Required: no
  • Allowed values: Recommended, Homepage navigation, All discovered

Run it

One call presses the Widget and answers the run once its status is succeeded or failed, however long it takes. The key is read from ABYSS_API_KEY. Install the library with npm install abysshub or pip install abysshub; cURL sends the Fields as a form.

JavaScript

// index.mjs
import Abyss from "abysshub";

const abyss = new Abyss();

const run = await abyss.run("widget_376", {
  website_url: "https://wordlift.io",
  pages: 16,
  discovery_mode: "Recommended",
});

console.log(run.result);
for (const path of await run.save("output")) {
  console.log(path);
}

Python

from abysshub import Abyss

abyss = Abyss()

run = abyss.run("widget_376", {
    "website_url": "https://wordlift.io",
    "pages": 16,
    "discovery_mode": "Recommended",
})

print(run.result)
for path in run.save("output"):
    print(path)

cURL

curl https://api.abysshub.com/v1/widgets/widget_376/runs \
  -H "Authorization: Bearer $ABYSS_API_KEY" \
  -F website_url=https://wordlift.io \
  -F pages=16 \
  -F discovery_mode=Recommended

A succeeded run carries its result and its output_files. Their URLs are signed for each read: re-read the run, never store a URL. A run still queued or running in the answer (after an hour, or when the API is busy) is read again at its Location with ?wait=true.

What it answers

The Example Run's result, the JSON the Widget wrote to output/output.json:

{
  "home": "https://wordlift.io/",
  "page_count": 16,
  "warnings": [
    "Sampled 16 public pages from 48 discovered URLs; review before publishing."
  ],
  "status": "✅ Generated **16 verified-fetched page links** from **https://wordlift.io/**. Review the output before publishing.\n\n- Sampled 16 public pages from 48 discovered URLs; review before publishing.\n\nThe OKF bundle is a **draft starter**. The graph is an export preview; nothing is written to a WordLift Knowledge Graph.",
…

More

Keys, files, errors and limits: API access.