Async jobs & pagination
Collect results from longer jobs and walk through paginated resources.
Asynchronous requests
Longer operations run as jobs. Synchronous scrape, map, parse, and extract requests may return HTTP 202 when they outlast the requested wait time. Batch scrapes and site crawls also expose job-status endpoints.
Poll for results
- Store the returned job ID and status URL.
- Poll the status URL with the same product API key.
- Read the returned status or HTTP response before treating the job as complete.
- Handle failed or cancelled jobs explicitly.
curl "https://factget.dev/crawl/v1/scrape/YOUR_JOB_ID" \
--header "Authorization: Bearer $FACTGET_CRAWL_KEY"See each endpoint for its response contract. The waiting-style and batch job endpoints do not all return the same shape.
Pagination
Collection and job-result endpoints may return a next URL. Follow that URL until it is null or absent. Preserve its cursor and filters; treat cursors as opaque values.
Only send your API key to the trusted FactGet origin. Check the endpoint’s query parameters for the allowed limit and cursor fields.