Firecrawl crawl limit: 10,000 default vs Sume's 20 pages

Firecrawl's crawl defaults to 10,000 pages. Sume's crawl takes limit up to 20 (default 10), so narrow with path prefixes and run several keyed crawls.

4 min readSume
All posts

Sume's crawl endpoint, POST /v1/firecrawl/crawl, accepts limit from 1 to 20 with a default of 10. Firecrawl's crawl guide says its default limit is 10,000 pages. A site section bigger than 20 pages needs several narrower crawls, or crawl_map to list URLs and crawl_scrape for the ones you want.

Firecrawl's numbers are from its crawl guide and Sume's from OpenAPI and MCP tool descriptions, read 2026-09-30.

How do the two crawl limits compare?

Both also differ in how you collect results.

Crawl limits and retrieval, Firecrawl guide vs Sume docs, read 2026-09-30.
ItemFirecrawlSume
Page limitDefault 10,000limit 1-20, default 10
StartCrawl requesturl plus a stable key (Idempotency-Key header over HTTP, idempotency_key in MCP)
CollectPoll GET /v2/crawl/{id}jobs_wait on the job id, then crawl_get
Result lifetime24 hours after completionNot stated in the sources read

How do I cover a site bigger than 20 pages?

Split it by section and give each crawl its own idempotency key, scoping each with include_paths (how they differ from Firecrawl's). Or call crawl_map (limit 1-100) for URLs and scrape the relevant ones.

How do I wait for a crawl?

The description says to call jobs_wait on the returned job_id, then crawl_get with the same id, and never to start a second crawl to poll. Reusing the key on a retry is what keeps a repeat from becoming a second crawl; see idempotency keys.

What does truncated mean?

crawl_get returns bounded pages, and truncated means the result was cut. The guidance is to narrow the scope if you need more content, which is the same fix as the 20-page limit: smaller prefixes, more crawls.

Sources

Related posts

More in Developers

All Developers posts

Written by Sume