factget
Request Access
Developer hub
Console
Developers /Crawl API

Batch scrape pages

Scrape many URLs as one async job; poll url (or use webhook).

POST/crawl/v1/batch/scrape
POST example
curl --request POST \
  --url 'https://factget.dev/crawl/v1/batch/scrape' \
  --header "Authorization: Bearer $FACTGET_CRAWL_KEY" \
  --header 'Content-Type: application/json' \
  --data '{
  "urls": [
    "https://example.com"
  ],
  "formats": [
    "markdown"
  ]
}'

Replace IDs and sample values with your own. Examples are not executed in your browser.

Authorization

Authorizationstring · header · required

Bearer Crawl API key with the required scope. Learn about authentication →

Request body

application/json
urlsstring[]required

The public source URLs for this job.

minItems: 1maxItems: 1000
originstring | null
maxLength: 200
formatsstring[]

Choose the output formats you need. Each item must be one of the allowed values.

Allowed: "markdown", "html", "rawHtml", "links", "extract"minItems: 1
onlyMainContentboolean

Return the main page content, excluding common navigation and page furniture.

Default: true
includeTagsstring[] | null

Include the matching HTML elements in the result.

maxItems: 50
excludeTagsstring[] | null

Exclude the matching HTML elements from the result.

maxItems: 50
waitForinteger

Additional time to wait for the page, in milliseconds.

Default: 0minimum: 0maximum: 30000
jsCodestring | null
maxLength: 10000
extractobject | null
Show nested properties
typestringrequired
Allowed: "css", "xpath", "regex"
schemaobject | null
ignoreInvalidURLsboolean
Default: false
webhookobject | null
Show nested properties
urlstringrequired

The public URL to read.

headersobject
metadataobject
eventsstring[]
Allowed: "started", "page", "completed", "failed"minItems: 1

Responses

200Successful Response
application/json

The published schema does not define individual response fields for this endpoint. Inspect the returned JSON in your integration.

422Validation Error
application/json
detailobject[]
Show item properties
locstring | integer[]required
msgstringrequired
typestringrequired
inputobject
ctxobject

The API schema lists the responses above. Long-running operations can also return 202; see async jobs. For application errors, see errors & retries.

FactGet developer documentationBack to API overview