Crawler / Page / Fetch

Convert a URL into clean Markdown—without a browser.

Send a public URL with method=fetch when its useful content is already in the returned HTML. AnyCrawler follows redirects and returns results.markdown, the final URL, status, and credits_used. Metadata, links, and media are optional. The base request costs 2 credits; each optional field group adds 1 credit.

Start Free Read Docs

Fast low-cost path

Send a URL and get Markdown back

For docs, articles, and other pages with useful HTML, send url and method=fetch. The response places the readable body in results.markdown, with title, final URL, HTTP status, and an estimated Markdown token count. Choose markdown for fuller extraction or readability for a leaner body that may omit some information.

To try one URL without an API key, use theFree extraction preview. It is a separate, cacheable preview and does not expose the authenticated API's method and output controls.

2-credit base requestFast HTTP retrievalFresh lightweight pathMarkdown normalized

curl -X POST "https://api.anycrawler.com/v1/crawl/page" \
  -H "Authorization: Bearer $ANYCRAWLER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/docs",
    "method": "fetch"
  }'

Send url and method=fetch to use the 2-credit base path.

{
  "ok": true,
  "requested_url": "https://example.com/docs",
  "canonical_url": "https://example.com/docs",
  "final_url": "https://example.com/docs",
  "status_code": 200,
  "credits_used": 2,
  "results": {
    "title": "Example Docs",
    "description": "Product documentation",
    "markdown": "# Example Docs\n\nInstall the SDK...",
    "markdown_tokens": 428
  }
}

The minimal request settles at 2 credits. Metadata, links, and media appear only when requested; each group adds 1 credit.

What you get

Readable content with a traceable URL

The fetch response places clean text in results.markdown. Use the final URL and status for source checks, and request metadata, links, or media only when your workflow needs them.

Markdown and token counts for agent context
Final URL and status code for traceability
Optional metadata, links, and media for enrichment
Credit and request fields for cost control

How it works

Fetch, clean, return structured fields

The gateway handles request validation, account checks, upstream fetch, response filtering, and credit settlement.

Submit the target URL

Send a POST request with url and method=fetch. Optional flags control metadata, links, media, and Markdown variant.

AnyCrawler fetches the page

The gateway validates your API key, checks credits, and retrieves the page through the lightweight fetch path.

HTML becomes Markdown

Noisy HTML is normalized into Markdown and token counts, with optional metadata and extracted links attached under results.

You get usage evidence

The response includes status, final URL, credits_used, and request headers for production debugging.

Use cases

Extract articles and documentation without launching a browser

Use Fetch when the page already contains the text you need in its HTML response.

Use

Documentation ingestion

Pull docs, changelogs, help centers, and API references into RAG pipelines without paying for browser execution on every page.

Use

Low-cost content monitoring

Check pricing, policy, product, or competitor pages frequently while keeping the base request cost at the lightweight fetch rate.

Use

Article and blog extraction

Convert long-form HTML pages into Markdown and token counts so research agents can read the useful body instead of raw markup.

Request fields

Keep the payload small, add fields only when you need them

The stable public contract focuses on the fields below. Undocumented passthrough fields are not part of new integrations.

Field	Type	Required	Notes
`url`	string	Yes	The public page URL to fetch and normalize.
`method`	"fetch"	Yes for fetch	Selects the lightweight fetch path. If omitted, method defaults to render.
`include_metadata`	boolean	No	Adds results.metadata when available. Defaults to false; +1 credit.
`include_links`	boolean	No	Adds results.links for follow-up crawl planning. Defaults to false; +1 credit.
`include_media`	boolean	No	Adds results.media when available. Defaults to false; +1 credit.
`markdown_variant`	"markdown" or "readability"	No	Defaults to markdown for fuller output. Readability may omit some page information; both return results.markdown at no extra cost.
`user_agent`	string or null	No	Optional paid-plan override for the outbound user agent. Empty strings and null are treated as unset.

Response fields

Fields your integration can rely on

AnyCrawler strips internal timing and debug fields before returning the public response.

Field	Meaning
`ok`	Whether the gateway completed the crawl successfully.
`requested_url`	The URL submitted in the request.
`canonical_url`	Canonical URL discovered from the page when present.
`final_url`	The final URL after redirects.
`status_code`	HTTP status code returned by the target page.
`credits_used`	Credits settled for the request.
`results.title / description / keywords`	Extracted page identity and descriptive fields when available.
`results.markdown`	Clean Markdown body normalized for AI context and downstream parsing.
`results.markdown_tokens`	Estimated token count for the Markdown payload.
`results.metadata`	Optional page metadata when include_metadata is true.
`results.links`	Optional extracted links when include_links is true.
`results.media`	Optional extracted media when include_media is true.
`error_code / error_message / retryable`	Failure details returned when ok is false; retryable indicates whether retrying can help.

const response = await fetch("https://api.anycrawler.com/v1/crawl/page", {
  method: "POST",
  headers: {
    authorization: `Bearer ${process.env.ANYCRAWLER_API_KEY}`,
    "content-type": "application/json",
  },
  body: JSON.stringify({
    url: "https://example.com/docs",
    method: "fetch",
    include_metadata: true,
  }),
});

const data = await response.json();

console.log(data.results?.markdown);
console.log(response.headers.get("x-request-id"));

Use the same stable public request fields from any runtime with standard HTTP support.

Fetch vs render

Use fetch for direct HTML. Use render for JavaScript pages.

Fetch reads the returned HTML without opening a browser or using the render cache. If the useful body appears only after JavaScript runs, send the same URL withmethod=render instead.

Mode	Best for	Base cost
`fetch`	Docs, articles, reference pages, static HTML, fresh content checks	2 credits
`render`	JavaScript apps, client-side routing, browser-dependent pages	10 credits

Errors

Status codes are explicit enough for production handling

Use response status plus error_code and request headers to route retries, credit issues, and authentication problems.

200Page fetched and normalized.

400Invalid JSON or unsupported request field value.

401Missing, invalid, or revoked API key.

402Account does not have enough credits.

403Account inactive or paid-plan-only field requested.

409Gateway could not reserve account capacity; retry the request.

413Request body exceeds the 256 KiB public gateway limit.

429Rate limit reached.

502Upstream connection failed.

503Database or upstream worker is not configured.

504Target or worker timed out.

Headers

Track every request

Successful and error responses include gateway metadata headers such as x-request-id, x-credits-reserved, and x-credits-used. Store them when you need support or cost audits.

Pricing

Fetch credit rules

Fetch uses 2 credits per successful request. include_metadata, include_links, and include_media each add 1 credit; readability adds none. Fetch does not use render cache-hit pricing.

A non-empty user_agent override requires a paid plan. For included credits and plan limits,compare all plans.

FAQ

Common fetch API questions

Short answers for teams deciding whether this is the right crawl mode.

When should I use fetch instead of render?

Use fetch for pages where the useful content is available in the returned HTML, such as docs, blogs, articles, reference pages, and many public content sites. Use render when JavaScript execution is required.

Does fetch still return Markdown?

Yes. The fetch path still normalizes output into results.markdown and results.markdown_tokens, so AI agents and RAG pipelines can consume cleaner text instead of raw HTML.

Can I get links and metadata in the same response?

Yes. Set include_metadata, include_links, or include_media to true. These optional fields are omitted by default to keep responses lean.

Does fetch use the render cache?

No. Fetch requests do not read from or write to the render cache, even if accept_cache is sent. Use render when JavaScript execution is needed.