Qcrawl API is live in production

The complete web data
platform.

Scrape any site. Drive any browser. Plug into any AI agent. Enrich any domain. One API key, one bill — covering everything most teams stitch together from three or four separate vendors.

Start free See live data Talk to sales

No credit card required. 1,000 requests free, every month.

Scrape & crawl

Any page, any site

Markdown, HTML, links, structured data, or vertical schemas. One URL in, clean data out.

Browser & automation

Drive the web

Action macros, JavaScript eval, captcha solving, plus a remote Chrome endpoint your Puppeteer code talks to.

AI & agents

MCP-native

Plug Qcrawl into Claude, Cursor, or any MCP-compatible agent in one config line. Give your agent the open web.

Intelligence

Every signal behind a domain

DNS, WHOIS, SSL, tech stack, contacts, email verification — same API, same call.

27+

Endpoints in production

Verticals covered

< 800 ms

Average response time

1,000

Free monthly requests

Real call. Real response.

This is an actual response from POST /v1/intel/page against a public domain. No screenshots, no mock data.

Request

POST https://api.qcrawl.com/v1/intel/page
Authorization: Bearer osk_…
Content-Type: application/json

{
  "url": "https://anthropic.com"
}

Response · 402 ms

{
  "status": "success",
  "url": "https://anthropic.com",
  "tech_stack": ["AWS", "Cloudflare"],
  "contacts": {
    "socials": { "twitter": "…", "linkedin": "…" },
    "emails": [],
    "phones": []
  },
  "logos": [
    { "type": "icon", "src": "/favicon.ico" },
    { "type": "apple-touch", "src": "/apple-icon.png" }
  ],
  "feeds": [],
  "time_ms": 402
}

DNS intelligence

email_provider: "Google Workspace"
nameserver_provider: "Cloudflare"
has_spf: true
has_dmarc: true

SSL certificate

issuer: "Let's Encrypt"
days_remaining: 51
sans: ["anthropic.com",
       "console.anthropic.com"]

WHOIS

registrar: "MarkMonitor, Inc."
created: "2001-10-02"
expires: "2033-10-02"
nameservers: ["isla.ns.cloudflare.com"]

Quickstart

First call in five minutes.

Get your API key from the dashboard, hit the endpoint, get clean data. The hardest part is choosing a URL.

curl -X POST https://api.qcrawl.com/v1/scrape \
  -H "Authorization: Bearer osk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "format": "markdown",
    "stealth": true
  }'

One API. Every job.

From a single page to a full site, from raw HTML to structured product data — the same key, the same patterns, everywhere.

Scraping core

POST /v1/scrapeSingle URL to markdown, HTML, text, or links.
POST /v1/scrape/smartAuto-route static pages to fast HTTP; dynamic pages to a full browser.
POST /v1/scrape/batchUp to 100 URLs in parallel, one response.
POST /v1/scrape/asyncQueue a single scrape, receive a webhook when ready.
POST /v1/scrape/batch/asyncFire-and-forget batches with no upper bound — webhook on completion.
GET /v1/jobs/{id}Poll status of any queued job.
POST /v1/captcha/solveSolve a reCAPTCHA v3 invisible challenge and return a token in under a second. v3 only — by design.
POST /v1/captcha/autoAuto-detect a reCAPTCHA v3 site key on a page and solve it in the same call.

Extraction & parsing

POST /v1/extract/cleanArticle body with no nav, ads, or sidebars. Reading time and word count included.
POST /v1/extract/structuredJSON-LD, Microdata, OpenGraph, and Twitter Card in one payload.
POST /v1/extract/contactsPull emails, phones, and social profile URLs from any text or HTML blob.
POST /v1/extract/tablesEvery <table> in an HTML blob as structured rows. Colspan/rowspan expanded.

Document conversion

POST /v1/convert/html-to-markdownRaw HTML to clean CommonMark. No fetch — you supply the HTML.
POST /v1/convert/pdf-to-markdownPDF → Markdown. Page-per-section. Scanned PDFs flagged for OCR.
POST /v1/convert/docx-to-markdownWord documents → Markdown. Headings, lists, tables preserved.
POST /v1/convert/xlsx-to-markdownExcel workbooks → Markdown tables. One section per sheet.
POST /v1/convert/pptx-to-markdownPowerPoint decks → Markdown. Slide titles, bullets, speaker notes.
POST /v1/convert/csv-to-markdownCSV text → Markdown table. Auto-detects delimiter.

Domain intelligence

POST /v1/dns/lookupA, MX, TXT, NS, CNAME, AAAA records.
POST /v1/dns/intelligenceEmail provider, nameserver provider, detected technologies.
POST /v1/intel/sslIssuer, expiry, SANs, days remaining.
POST /v1/intel/whoisRegistrar, creation date, expiry date, contact emails.
POST /v1/intel/pageTech stack, social links, emails, phones, logos, feeds.
POST /v1/intel/geoipCountry, region, city, lat/lon, timezone, ASN and ISP for any public IP.
POST /v1/intel/geoip/bulkUp to 100 IPs per call — same enrichment, one credit.
POST /v1/intel/asnOperator name and every routed CIDR prefix for an Autonomous System Number.
POST /v1/intel/headersResponse headers + security-header grade (HSTS, CSP, etc.) A–F.
POST /v1/intel/redirectsTrace the full redirect chain hop by hop.
POST /v1/intel/uptimeIs the URL up? Status, latency, SSL expiry warning.
POST /v1/intel/email-authSPF + DKIM + DMARC + BIMI parsed and graded A–F.
POST /v1/intel/blacklistCheck IPs or domains against six major DNSBLs in parallel.
POST /v1/intel/dns-propagationSame record across nine resolvers — has your DNS change propagated?
POST /v1/intel/ip-reputationDatacenter / cloud range / Tor exit classification for any IP.
POST /v1/verify/emailSMTP handshake, MX check, catch-all and disposable detection.

Parsing & conversion

POST /v1/parse/urlDecompose URLs: scheme, host, query params, public suffix, registrable domain.
POST /v1/parse/useragentUser-Agent string to browser, OS, device, and a device-kind label.
POST /v1/parse/emailSyntax + role/disposable/free-provider flags. No SMTP probe — the cheap, instant tier.
POST /v1/convert/html-to-markdownRaw HTML to clean CommonMark. No fetch — you supply the HTML.

AEO & SEO

POST /v1/aeo/ai-bot-allowlistWhich AI crawlers (GPTBot, ClaudeBot, PerplexityBot, …) does this site allow?
POST /v1/aeo/llms-txt-auditValidate /llms.txt against the AEO-readiness convention. Score + link-rot check.
POST /v1/aeo/schema-coverageJSON-LD / Microdata / OG coverage scored against the 12 AEO-critical schema.org types.
POST /v1/aeo/citation-readinessPost-ChatGPT EEAT: numerical specifics, named entities, authoritative citations, byline, last-updated.
POST /v1/aeo/llm-fetch-simulatorWhat does GPTBot actually see? 12 parallel UA probes. Detects JS-only + cloaking.
POST /v1/aeo/page-auditPremium: 9-probe consolidated AEO scorecard with ranked recommendations.
POST /v1/aeo/competitor-diffPremium: page-audit head-to-head across 2–5 URLs. The consultant invoice line item.
POST /v1/seo/meta-auditGrade title, meta description, canonical, H1, OG tags A–F.
POST /v1/seo/readabilitySix standard readability formulas plus reading time. Pure transform.
POST /v1/seo/snippet-candidatesParagraph / list / table candidates for featured snippets from any HTML blob.

Crawling

POST /v1/crawlFull-site crawler with depth, budget, and robots.txt controls.
POST /v1/intel/sitemapUnroll any sitemap.xml, including nested index sitemaps.
POST /v1/intel/robotsCheck whether a path is allowed for a given user agent.

Frequently asked questions

What kinds of sites can Qcrawl handle? ▾

Qcrawl handles static pages, JavaScript-heavy single page applications, e-commerce sites, real estate listings, maps, and most content behind anti-bot defenses. For sites with the strongest protection — Amazon, Zillow, certain travel and ticketing sites — pairing your request with a residential proxy improves success rates significantly.

How is Qcrawl different from Apify, ScraperAPI, or Bright Data? ▾

Apify pioneered the marketplace model for scrapers and ScraperAPI nailed developer ergonomics. Qcrawl takes the same simplicity and goes further on two fronts: every extractor is maintained in-house so reliability stays consistent across the catalog, and a complete domain intelligence layer — DNS, WHOIS, SSL, tech stack, contacts — ships in the same API at no extra cost.

Do you charge for failed requests? ▾

No. Failed requests do not count against your monthly quota. You only pay for successful responses.

What does the free tier include? ▾

1,000 requests every month, every endpoint, no credit card. Plenty to evaluate the API end to end and run real prototypes.

How do I scrape Amazon or Zillow specifically? ▾

Use the dedicated actor endpoints — /v1/actors/amazon and /v1/actors/zillow. They return structured product or property data in a clean JSON schema. Both sites use aggressive anti-bot defenses, so we recommend supplying a residential proxy parameter for production workloads.

Can Qcrawl scrape pages that require login? ▾

Yes, via the actions array on /v1/scrape. You can drive clicks, typing, waits, and form submissions natively. For pages that need persistent sessions, talk to us about the enterprise tier.

What is the average response time? ▾

Static pages typically return in under 800 milliseconds. JavaScript-rendered pages take 1.5 to 4 seconds depending on complexity. Domain intelligence calls return in 50 to 500 milliseconds.

Is the API rate-limited? ▾

Yes. Each plan has a requests-per-minute ceiling and a monthly request quota. Rate limit headers are returned with every response so your client can adapt in real time.

Start pulling clean data in minutes.

1,000 requests free every month. No credit card required.

Start free Talk to sales

The complete web data platform.