How to Convert a Web Page to Clean Markdown for an LLM or AI Agent

To convert a web page to Markdown for an LLM, fetch the HTML, keep only the main content, and convert that to Markdown. You can do all three steps yourself with a few lines of Node.js, or call a hosted reader that does them for you with one GET. Which one fits depends on how much control you need and what the page looks like.

This guide shows both, with output from real runs on October 29, 2026, and lists the cases where neither is enough. The tool details come from each project's own documentation, checked on the same date.

What "clean" means for an LLM

A good conversion for a model or retrieval pipeline does four things:

• Drops page chrome. Navigation, footers, cookie banners, and ads cost tokens and add noise to retrieval.
• Keeps structure. Headings, lists, code blocks, and tables should survive, because they carry meaning and give you natural chunk boundaries.
• Keeps links usable. Relative links need resolving against the page URL so the agent can follow or cite them.
• Stays small. Smaller input means lower cost and more room for the rest of the prompt.

No converter is perfect at all four, and the output differs by page. Treat conversion as a step to inspect, not one to trust blindly.

What we measured

We converted one documentation page, https://react.dev/learn, with three methods on October 29, 2026. Sizes are bytes of output, not tokens, and this is a single page, so read it as an illustration rather than a benchmark.

All three conversions shrank the page by more than an order of magnitude, because most of the HTML is markup and scripts rather than prose. On a second page, the English Wikipedia article on Markdown, the raw HTML was 321,954 bytes and the ReplyNodes output was 48,882 bytes. We did not compare answer quality, so these numbers say nothing about which output makes a model answer better.

Looking at the outputs, each had small leftovers. The ReplyNodes output began with a Copy page Copy button label from the page. The Readability script left empty heading-anchor links such as [](#components "Link for Creating and nesting components "). Jina prepended Title:, URL Source: and Markdown Content: lines, which is useful metadata but is not part of the page. None of this is a defect to panic over, but it is why you should read a sample of your own pages through whichever tool you pick.

Option 1: Ask the origin for Markdown

Some sites can serve Markdown directly. Cloudflare's Markdown for Agents lets a site owner enable conversion at the edge, so a client that sends Accept: text/markdown gets Content-Type: text/markdown back. Cloudflare documents it as available on Pro, Business, and Enterprise plans at no cost, limited to 6 MiB of decompressed HTML, and only for HTML pages.

It is opt-in per site, so it is a cheap first try rather than something to rely on:

curl -sS -H "Accept: text/markdown" https://example.com/some-page

When we sent that header to react.dev/learn, the response was text/plain, not text/markdown, so the site was not serving converted Markdown for that header. Check the Content-Type of the response before you assume you got Markdown.

Option 2: Self-host with Readability and Turndown

If you need control over fetching, rules, and data handling, combine two small libraries. Mozilla Readability extracts the main article, as Firefox Reader View does. Turndown converts HTML to Markdown.

npm install @mozilla/readability jsdom turndown

// extract.mjs
import { JSDOM } from "jsdom"
import { Readability } from "@mozilla/readability"
import TurndownService from "turndown"
 
const url = process.argv[2]
const res = await fetch(url, { headers: { "User-Agent": "markdown-example/1.0" } })
if (!res.ok) throw new Error(`HTTP ${res.status} for ${url}`)
const html = await res.text()
 
const dom = new JSDOM(html, { url }) // url lets relative links resolve
const article = new Readability(dom.window.document).parse()
if (!article) throw new Error("Readability found no main content")
 
const turndown = new TurndownService({ headingStyle: "atx", codeBlockStyle: "fenced" })
console.log(`# ${article.title}\n\n${turndown.turndown(article.content)}`)

node extract.mjs https://react.dev/learn > page.md

We ran this with @mozilla/readability 0.6.0, jsdom 30.1.3, and turndown 7.2.4. A few details from the projects' own documentation matter in production:

• Pass the page URL to JSDOM so relative links and images resolve to absolute URLs.
• Readability does not sanitize its input, and its README recommends DOMPurify for untrusted content. Keep jsdom script execution at its default of disabled.
• Pages with very little text can return no result, since Readability skips articles below a character threshold (500 by default). The script above throws in that case instead of returning an empty file.
• Turndown defaults to setext headings and indented code blocks, which is why the script sets atx and fenced, the forms most LLM tooling expects.

This path costs you maintenance, and it fetches the page with a plain HTTP request, so it does not run JavaScript.

The script fetches whatever URL it is given. That is fine for a local run on pages you chose. If you turn it into a service that accepts URLs from users or from a model, validate the target first: allow only http and https, and refuse hosts that resolve to loopback, private, or link-local addresses. Otherwise a caller can use your server to reach internal resources (server-side request forgery).

Option 3: A keyless hosted reader

A hosted reader fetches and converts for you. Two are free to try without an account.

ReplyNodes Markdown takes the target as a path suffix:

curl -sS https://md.replynodes.com/https://react.dev/learn

Its own instructions describe it as free with no signup and no API key, returning text/markdown; charset=utf-8. The responses we saw carried x-ratelimit-limit: 100 and x-ratelimit-reset headers, so read those headers rather than assuming the limit is unlimited. The Markdown API page covers the endpoint.

Jina Reader works by prefix:

curl -sS https://r.jina.ai/https://react.dev/learn

Jina's Reader page says basic usage needs no key, and lists 20 requests per minute without one. Its default engine runs a headless browser, so client-side JavaScript executes before extraction, and a direct engine skips that for speed. Jina says it respects site access controls and does not bypass anti-bot protection.

Both returned HTTP 200 for the page above. They are quick for prototypes and low-volume agents. For heavier use, check each service's current limits and terms, since they can change.

Option 4: A scraping API

When you need scripted browser actions, structured extraction, or crawling a whole site, a scraping API covers more than conversion. Firecrawl's scrape documentation describes a POST /v2/scrape endpoint that returns Markdown or structured data, handles JavaScript-rendered pages, and says you can start without an API key, with a key raising rate limits. Each scrape costs one credit there, with extra credits for options such as JSON mode. If you need that range, see our comparison of Firecrawl alternatives and the guide to scraping, crawling, and mapping.

Where conversion fails

• JavaScript-rendered pages. A plain fetch sees the HTML before scripts run. If the content is built in the browser, the Readability script gets an empty shell. Use a reader or API that renders pages, and confirm that it does.
• Blocks and paywalls. Services that respect access controls will not get past a login or a bot wall. A request that fails can show up as an error or a thin page, so check status and length.
• Errors from the target. In our tests, md.replynodes.com returned a JSON error with HTTP 400 and "reason":"target_not_allowed" for an unresolvable host, and a plain 502 for a target that answered with HTTP 403. Handle non-200 responses and non-Markdown content types in your code.
• Untrusted content. Fetched text can contain instructions aimed at your model. Pass it as data, keep it out of the system prompt, and do not let it trigger actions on its own.
• Licensing. Converting a page does not change who owns it. Check the site's terms and robots rules for how you may use the content.

How to choose

• Start with Option 1 if the sites you read are likely to support it, and fall back when they do not.
• Use Option 2 when you need control over fetching or fixed data handling. For pages behind your own login, add the needed headers or cookies to the fetch call, which the example above does not do.
• Use Option 3 for prototypes and light agent use on public pages.
• Use Option 4 for rendered pages, crawling, or structured extraction at scale.

If your agent needs search and several kinds of public data rather than one-off conversion, read about the web scraping API for AI agents. For the keyless endpoint alone, the Markdown API is the shortest path.

FAQ

Does this work for JavaScript-rendered pages? Not with a plain fetch. The self-hosted script above reads server-delivered HTML only. Jina and Firecrawl document rendering options; we did not test JavaScript-heavy pages for any tool here, so test your own targets.

Is the keyless endpoint unlimited? No. md.replynodes.com reports its limit in response headers (we saw x-ratelimit-limit: 100), and Jina lists 20 requests per minute without a key. Read the headers and plan for limits.

Should I self-host or use a hosted reader? Self-host when you need control over fetching, rules, or data handling, or when you must fetch pages with your own credentials. Use a hosted reader when you want a working result in one request and your volume is modest.

Can I use converted content however I like? Not automatically. Conversion changes the format, not the rights. Check the source site's terms before storing or republishing its content.