Idempotency
Safely retry async-creating endpoints without queuing duplicate jobs or double-charging credits.
/v1/scrape/async Three endpoints accept an Idempotency-Key header to make POST retries safe: /v1/scrape/async, /v1/scrape/batch/async, and /v1/crawl. Pass any unique string (UUIDs are the convention) up to 255 characters. The first request runs normally; any retry within 24 hours that carries the same key returns the original response verbatim, with an Idempotency-Replayed: true response header. No second job is queued. No second credit is charged. Sending the same key with a different request body returns 409 Conflict — that is almost always a client bug worth surfacing.
/v1/scrape/async, /v1/scrape/batch/async, /v1/crawl — every endpoint that creates a billable async job.
Synchronous endpoints (/v1/scrape, /v1/extract/*, /v1/intel/*, etc.) do not need idempotency: they return the full result inline, so a retry just gets you the same answer again at the cost of one credit. Use a cache on your side if that is a concern.
Parameters
| Name | Type | Required | Default | Description |
|---|---|---|---|---|
| Idempotency-Key | header | no | — | Any string up to 255 characters. UUIDv4 is the recommended format. Scope: per user, per endpoint. |
Request
# First request: runs normally, returns a job_id.
curl -X POST https://api.qcrawl.com/v1/scrape/async \
-H "Authorization: Bearer osk_..." \
-H "Idempotency-Key: a3f6c8d2-1e9b-4c7a-9f10-8e2d4b6c1234" \
-d '{"url": "https://example.com", "webhook_url": "https://yourapp.com/cb"}'
# Retry with the same key (within 24 h): returns the SAME job_id,
# no second job queued, no second credit charged.
# Response carries header: Idempotency-Replayed: true
curl -X POST https://api.qcrawl.com/v1/scrape/async \
-H "Authorization: Bearer osk_..." \
-H "Idempotency-Key: a3f6c8d2-1e9b-4c7a-9f10-8e2d4b6c1234" \
-d '{"url": "https://example.com", "webhook_url": "https://yourapp.com/cb"}'Response
{
"status": "queued",
"job_id": "c5584f4b-07f8-4329-8484-44eebf6b6e48"
}
// On the replay, identical body with response header:
// Idempotency-Replayed: trueErrors
| Code | Meaning |
|---|---|
| 409 | Idempotency-Key was used with a different request body. Use a fresh key for a new request. |