factget
Request Access
Developer hub
Console
Developers /Crawl API

Scrape a page

Scrape one URL. Waits up to timeout ms; past it, 202 {id, url}.

POST/crawl/v1/scrape
POST example
curl --request POST \
  --url 'https://factget.dev/crawl/v1/scrape' \
  --header "Authorization: Bearer $FACTGET_CRAWL_KEY" \
  --header 'Content-Type: application/json' \
  --data '{
  "url": "https://example.com",
  "formats": [
    "markdown",
    "links"
  ],
  "onlyMainContent": true
}'

Replace IDs and sample values with your own. Examples are not executed in your browser.

Authorization

Authorizationstring · header · required

Bearer Crawl API key with the required scope. Learn about authentication →

Request body

application/json
urlstringrequired

The public URL to read.

originstring | null
maxLength: 200
formatsstring[]

Choose the output formats you need. Each item must be one of the allowed values.

Allowed: "markdown", "html", "rawHtml", "links", "extract"minItems: 1
onlyMainContentboolean

Return the main page content, excluding common navigation and page furniture.

Default: true
includeTagsstring[] | null

Include the matching HTML elements in the result.

maxItems: 50
excludeTagsstring[] | null

Exclude the matching HTML elements from the result.

maxItems: 50
waitForinteger

Additional time to wait for the page, in milliseconds.

Default: 0minimum: 0maximum: 30000
jsCodestring | null
maxLength: 10000
extractobject | null
Show nested properties
typestringrequired
Allowed: "css", "xpath", "regex"
schemaobject | null
timeoutinteger

How long to wait for the operation, in milliseconds, before returning a job to poll.

Default: 60000minimum: 1000maximum: 120000

Responses

200Successful Response
application/json

The published schema does not define individual response fields for this endpoint. Inspect the returned JSON in your integration.

422Validation Error
application/json
detailobject[]
Show item properties
locstring | integer[]required
msgstringrequired
typestringrequired
inputobject
ctxobject

The API schema lists the responses above. Long-running operations can also return 202; see async jobs. For application errors, see errors & retries.

FactGet developer documentationBack to API overview