Crawler / Search / Scholar

Discover research papers for your next literature review.

Use Scholar Search to build a shortlist of research-paper pages and other scholarly sources for a literature review. Send channel=scholar to discover paper candidates with links and snippets when available. Citation details may be absent; search does not provide full paper text or verify peer review status.

Start Free Read Docs

Direct answer

What is a scholar search API for?

AnyCrawler gives research agents scholarly result candidates from a technical query. Build a paper shortlist, then visit accessible links for abstracts, author details, and source text as available.

Scholarly resultsCitation context when availableResearch discoveryReference candidates

Integration fit

Use scholar search for literature discovery

Rank links by relevance, use citation context when present, then read accessible abstracts or source pages before citing a paper in a summary.

Search before crawl

Find candidates before spending crawl work

Search keeps agents from crawling blindly. Start from a query, pick the right channel, and return normalized result collections that can become crawl targets, review queues, or research inputs. Use the search hub to compare channels before choosing a specific vertical.

Scholarly resultsCitation context when availableResearch discoveryReference candidates

curl -X POST "https://api.anycrawler.com/v1/search" \
  -H "Authorization: Bearer $ANYCRAWLER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "channel": "scholar",
    "query": "web automation accessibility testing",
    "country": "us",
    "page": 1,
    "results_per_page": 10
  }'

Use the unified /v1/search endpoint and select the search type with the channel field.

Use cases

Scholar Search API for Research Papers use cases

Rank links by relevance, use citation context when present, then read accessible abstracts or source pages before citing a paper in a summary.

Use

Seed a literature review

Start with a technical query and collect paper candidates before crawling abstracts, author pages, or references.

Use

Support research agents

Give agents scholarly candidate links to inspect when a task needs papers, technical references, or prior work.

Use

Trace technical claims

Find citation-oriented sources, then crawl the strongest references into Markdown for synthesis or review.

How it works

Query, normalize, route the strongest results

The gateway handles request validation, account checks, search provider routing, response normalization, and credit settlement.

Submit a query

Send a POST request with query, channel=scholar, and optional localization or pagination fields.

AnyCrawler routes the search

The gateway validates your API key, checks credits, and sends the request to the selected search collection.

Results are normalized

Upstream results are returned in a predictable response envelope with channel, query, nested search parameters, status, and credits.

You choose the next step

Use result links as crawl targets, visual review candidates, research inputs, or source lists for a larger agent workflow.

{
  "ok": true,
  "query": "web automation accessibility testing",
  "credits_used": 2,
  "status_code": 200,
  "results": {
    "search_parameters": {
      "channel": "scholar",
      "country": "us",
      "page": 1,
      "query": "web automation accessibility testing",
      "results_per_page": 10
    },
    "organic": [
      {
        "position": 1,
        "title": "Automated browser testing for accessibility",
        "link": "https://example.com/research/accessibility-browser-testing",
        "snippet": "A scholarly result candidate for citation-oriented research."
      }
    ]
  }
}

Result collections vary by channel, but the public response envelope stays consistent.

What you get

Search results that can become crawl targets

The scholar search response is built for routing. Use snippets for triage, links for follow-up crawl requests, and status and credit fields for production observability.

Research-oriented source candidates
Citation context when the provider supplies it
Technical references for deeper crawling
Candidate paper pages for fetch or render extraction

Request fields

One search endpoint, explicit channel selection

The stable public contract focuses on the fields below. Undocumented passthrough fields are not part of new integrations.

Field	Type	Required	Notes
`channel`	"page", "images", "news", "videos", or "scholar"	Yes	Selects the search vertical on the unified public search endpoint.
`query`	string	Yes	A non-empty search query, up to 512 characters.
`country`	string or null	No	Optional localization hint for the upstream provider, not a strict country filter; up to 128 characters.
`page`	integer	No	Logical result page from 1 to 100. Defaults to 1 and combines with results_per_page.
`results_per_page`	integer	No	Maximum results from 1 to 100. Defaults to 10; consecutive 10-result provider pages stop early on a short page.

Response fields

Fields your integration can rely on

AnyCrawler keeps the search envelope consistent while the result collection reflects the selected channel.

Field	Meaning
`ok`	Whether the gateway completed the search successfully.
`query`	The query submitted in the request.
`credits_used`	Credits settled at 2 per primary result actually returned.
`status_code`	The upstream search provider status code returned through the gateway.
`results.search_parameters.channel`	The search vertical used for the request.
`results.search_parameters.country`	The country hint used for localized search results.
`results.search_parameters.query`	The query mirrored inside the normalized search parameters.
`results.search_parameters.page`	The search result page returned.
`results.search_parameters.results_per_page`	The requested maximum number of primary results.
`results.organic`	Organic scholarly result candidates for research and technical-reference workflows.
`results.knowledge_graph / answer_box / people_also_ask / related_searches`	Optional normalized provider collections when they are available for the query.
`error_code / error_message / retryable`	Failure details returned when ok is false; retryable indicates whether retrying can help.

const response = await fetch("https://api.anycrawler.com/v1/search", {
  method: "POST",
  headers: {
    "authorization": `Bearer ${process.env.ANYCRAWLER_API_KEY}`,
    "content-type": "application/json",
  },
  body: JSON.stringify({
    channel: "scholar",
    query: "web automation accessibility testing",
    country: "us",
    page: 1,
    results_per_page: 10,
  }),
});

const data = await response.json();
console.log(data.results);

Change channel to switch between page, images, news, videos, and scholar search.

Search vs crawl

Use scholar search for literature discovery

Rank links by relevance, use citation context when present, then read accessible abstracts or source pages before citing a paper in a summary. Where a result links to an accessible web page, fetch or render can extract that page's Markdown and metadata. Screenshot is a separate request for visual evidence; page crawling does not download media or bypass source access restrictions.

API	Best for	Returns
`search`	Finding candidate pages, sources, media, news, videos, or papers	Search result collections
`fetch`	Fast, low-cost extraction when useful content is already in HTML	Markdown and page fields
`render`	JavaScript-loaded pages that need browser execution	Rendered Markdown and page fields
`screenshot`	A separate visual record of a selected public page	Stored image fields

Errors

Status codes are explicit enough for production handling

Use response status plus error_code and request headers to route retries, credit issues, authentication problems, and upstream provider failures.

200Search completed and normalized.

400Invalid JSON, missing channel, missing query, or unsupported field value.

401Missing, invalid, or revoked API key.

402Account does not have enough credits.

403Account inactive.

409Gateway could not reserve account capacity; retry the request.

413Request body exceeds the 256 KiB public gateway limit.

429Rate limit reached.

502Upstream provider connection failed or returned an unexpected response.

503Database or search provider is not configured.

504Search provider or gateway timed out.

Billing

Predictable result-count pricing

Search reserves for the requested maximum, then settles only the primary results actually returned and refunds unused reserved credits.

Pricing

Search result pricing

Search settles at 2 credits per primary result actually returned. A request for 10 results costs 20 credits if all 10 return; a short result list costs less. Fetching or rendering a selected page is billed separately.

See included credits and request limits on thefull pricing page.

FAQ

Common scholar search questions

Short answers for teams using search to choose better crawl targets.

Can I use the same endpoint for other search types?

Yes. POST to /v1/search and change the channel field to page, images, news, videos, or scholar. Each channel also has this dedicated theme page for integration planning.

How do search results fit with page crawling?

Use search to find candidate URLs, then call fetch or render on accessible web pages for Markdown, metadata, or links. Screenshot is a separate request when visual evidence is needed.

Does scholar search return full paper text?

No. Scholar search returns discovery candidates and source context. Use accessible result links with fetch or render when you need text extraction, subject to the source's access rules.

How can research agents use scholar results?

Shortlist relevant links, use citation details if supplied, deduplicate papers, and inspect accessible sources before adding them to an evidence set.