> ## Documentation Index
> Fetch the complete documentation index at: https://docs.adscrawl.net/llms.txt
> Use this file to discover all available pages before exploring further.

# Extract SPA Data

> Extract structured data from JavaScript-heavy SPAs using custom CSS or network fields, built-in templates, or inspect mode to discover extraction candidates.

Submit a page URL and extraction configuration to retrieve structured data from a JavaScript-rendered SPA. You can use custom field definitions, a built-in template, or `inspect` mode to discover what data is available on a page.

<ParamField body="url" type="string">
  Full HTTP(S) URL for the target SPA using port 80 or 443. Required for most templates. For `google-trends-explore`, prefer the `keyword` field instead. SimilarWeb requires a full page URL, not a bare domain.
</ParamField>

<ParamField body="keyword" type="string">
  Recommended for `google-trends-explore`. Provide 1 to 5 unique, non-empty comma-separated keywords, each at most 100 Unicode characters (for example, `"adspower,playwright"`). The server builds the canonical Trends URL automatically. Explicit `null`, numeric, or empty-string values return `INVALID_KEYWORD` and never fall back to `url`.
</ParamField>

<ParamField body="mode" type="string">
  Extraction mode. Defaults to `"extract"`.

  | Value | Behaviour |
  | - | - |
  | `"extract"` | Runs field or template extraction and returns structured `data`. |
  | `"inspect"` | Returns DOM and network candidates discovered on the page along with a `suggestedPlan` you can copy into a future extract request. |
</ParamField>

<ParamField body="template" type="string">
  Built-in site template ID. When provided, `fields` are supplied by the template rather than your request. Available template IDs:

  * `"similarweb-overview"` — website traffic, engagement, and ranking metrics.
  * `"google-trends-explore"` — interest-over-time data and relative averages for up to 5 keywords.
  * `"chrome-web-store-app-info"` — extension metadata from the Chrome Web Store (with CRX manifest fallback).
</ParamField>

<ParamField body="parameters" type="object">
  Template-specific parameters declared by the selected template. Refer to the template's `input` specification from `GET /spa-extract/templates`.
</ParamField>

<ParamField body="waitUntil" type="string">
  Navigation event to wait for before extraction. Custom extraction defaults to `"domcontentloaded"`. Site templates use the request value first, then the template default, then `"domcontentloaded"`.

  | Value | Behaviour |
  | - | - |
  | `"domcontentloaded"` | Waits for DOMContentLoaded — recommended for most SPAs. |
  | `"load"` | Waits for `window.load` including images and stylesheets. |
  | `"networkidle"` | Waits until no network connections for 500 ms. May timeout on long-polling sites. |
</ParamField>

<ParamField body="waitFor" type="object">
  Wait for a specific element or text to appear after SPA navigation and actions complete. `selector` and `text` may be combined.

  <Expandable title="waitFor fields">
    <ParamField body="selector" type="string">Wait for the first matching element to become visible.</ParamField>
    <ParamField body="text" type="string">Wait for the first element containing this text to become visible.</ParamField>
    <ParamField body="timeoutMs" type="number">Timeout in milliseconds. Defaults to 15,000 and never exceeds the remaining task timeout.</ParamField>
  </Expandable>
</ParamField>

<ParamField body="actions" type="array">
  Optional page interactions executed in array order before extraction. Each step is capped at 30 seconds.

  <Expandable title="action types">
    <ParamField body="wait" type="object">`{ type: "wait", milliseconds: number }` — Pause for 0 to 30,000 ms.</ParamField>
    <ParamField body="waitForSelector" type="object">`{ type: "waitForSelector", selector: string, timeoutMs?: number }` — Wait for an element to become visible.</ParamField>
    <ParamField body="click" type="object">`{ type: "click", selector: string }` — Click the first matching element.</ParamField>
    <ParamField body="fill" type="object">`{ type: "fill", selector: string, value: string }` — Clear and fill the first matching input.</ParamField>
    <ParamField body="press" type="object">`{ type: "press", selector: string, key: string }` — Press a key on the first matching element.</ParamField>
    <ParamField body="scroll" type="object">`{ type: "scroll", selector?: string, x?: number, y?: number }` — Scroll an element into view or scroll the page. `y` defaults to 800.</ParamField>
  </Expandable>
</ParamField>

<ParamField body="fields" type="object">
  Record of named field definitions for `extract` mode. Each key becomes a property in the response `data` object. Use this when you are not using a template.

  <Expandable title="field definition">
    <ParamField body="source" type="string" required>`"dom"` to read from the page HTML, or `"network"` to read from intercepted JSON responses.</ParamField>
    <ParamField body="selector" type="string">CSS selector for a DOM field.</ParamField>
    <ParamField body="value" type="string">DOM read mode: `"text"` (default), `"html"`, or `"attribute"`.</ParamField>
    <ParamField body="attribute" type="string">Attribute name, used when `value` is `"attribute"`.</ParamField>
    <ParamField body="urlIncludes" type="string">Substring to match against a response URL for a network field.</ParamField>
    <ParamField body="path" type="string">JSON path within the matched network response, for example `$.data.metrics[0].value`.</ParamField>
    <ParamField body="multiple" type="boolean">Whether a DOM field should return all matching elements as an array.</ParamField>
    <ParamField body="parse" type="string">Coerce the value to `"string"`, `"number"`, `"integer"`, `"boolean"`, or `"json"`.</ParamField>
    <ParamField body="regex" type="string">Optional regular expression. When a capture group is present, group 1 is used.</ParamField>
    <ParamField body="required" type="boolean">Missing required fields return `422 SPA_REQUIRED_FIELDS_MISSING`.</ParamField>
  </Expandable>
</ParamField>

<ParamField body="schema" type="object">
  Legacy alias for `fields`, used only when `fields` is absent. String values are treated as DOM CSS selectors. Does not validate the response JSON and does not return `schemaValid` or `schemaErrors` fields. Prefer `fields` for all new integrations.
</ParamField>

<ParamField body="timeoutMs" type="number">
  Maximum time to wait for the task to complete, in milliseconds. Must be a positive integer no greater than `3,600,000`.
</ParamField>

<ParamField body="viewport" type="object">
  Viewport dimensions used when rendering the page.

  <Expandable title="viewport fields">
    <ParamField body="width" type="number">Viewport width in pixels.</ParamField>
    <ParamField body="height" type="number">Viewport height in pixels.</ParamField>
  </Expandable>
</ParamField>

<ParamField body="locale" type="string">
  Browser locale, for example `"en-US"`. May follow trusted proxy metadata when omitted.
</ParamField>

<ParamField body="timezoneId" type="string">
  IANA timezone identifier, for example `"Asia/Shanghai"`. May follow trusted proxy metadata when omitted.
</ParamField>

<ParamField body="geolocation" type="object">
  Geolocation coordinates exposed to the page via the Geolocation API.

  <Expandable title="geolocation fields">
    <ParamField body="latitude" type="number">Latitude in decimal degrees.</ParamField>
    <ParamField body="longitude" type="number">Longitude in decimal degrees.</ParamField>
  </Expandable>
</ParamField>

<ParamField body="proxy" type="object">
  Custom proxy configuration. Cannot be combined with `countryCode`.

  <Expandable title="proxy fields">
    <ParamField body="server" type="string">Full proxy URL such as `http://host:port` or `socks5://host:port`. Cannot be combined with `host`.</ParamField>
    <ParamField body="protocol" type="string">`"http"` or `"socks5"`. Used in the split form.</ParamField>
    <ParamField body="host" type="string">Proxy host. Used in the split form.</ParamField>
    <ParamField body="port" type="number | string">Port from 1 to 65535.</ParamField>
    <ParamField body="username" type="string">Proxy username. Must be supplied together with `password`.</ParamField>
    <ParamField body="password" type="string">Proxy password. Must be supplied together with `username`.</ParamField>
  </Expandable>
</ParamField>

<ParamField body="countryCode" type="string">
  Managed residential proxy region. Cannot be combined with `proxy`.

  * `"GLOBAL"` — dynamic exit from 15 popular regions.
  * Two-letter country code — prefers a trusted proxy with dynamic fallback.
  * Omitted — a random trusted proxy is selected automatically.
</ParamField>

<ParamField body="userAgentMode" type="string">
  `"random"` lets the server pick a User-Agent from its library. Requests without an explicit `userAgent` default to `"random"`.
</ParamField>

<ParamField body="userAgentOs" type="string">
  Operating system used when `userAgentMode` is `"random"`. Accepted values: `"windows"` (default) or `"macos"`.
</ParamField>

<ParamField body="userAgent" type="string">
  Explicit User-Agent string. Overrides the random selection.
</ParamField>

<ParamField body="fingerprint" type="object">
  Browser fingerprint settings. When omitted, every signal defaults to random while keeping OS, GPU, CPU, memory, fonts, and device signals coherent.

  <Expandable title="fingerprint fields">
    <ParamField body="webRtc" type="string">`"forward"`, `"real"`, or `"disabled"`.</ParamField>
    <ParamField body="webGl" type="string">`"random"` or `"real"`. Controls WebGL vendor and renderer metadata.</ParamField>
    <ParamField body="webGpu" type="string">`"random"`, `"real"`, or `"disabled"`.</ParamField>
    <ParamField body="webGlImage" type="string">`"random"` or `"real"`. Controls WebGL image noise.</ParamField>
    <ParamField body="canvas" type="string">`"random"` or `"real"`. Controls canvas noise.</ParamField>
    <ParamField body="audioContext" type="string">`"random"` or `"real"`. Controls audio fingerprint noise.</ParamField>
    <ParamField body="clientRects" type="string">`"random"` or `"real"`. Controls layout measurement noise.</ParamField>
    <ParamField body="speechVoices" type="string">`"random"` or `"real"`. Returns an OS-matched speech voice list.</ParamField>
    <ParamField body="fonts" type="string">`"random"` or `"real"`. Returns an OS-matched font list.</ParamField>
    <ParamField body="hardware" type="string">`"random"` or `"real"`. Generates a coherent CPU thread count and memory pair.</ParamField>
    <ParamField body="doNotTrack" type="string">`"random"`, `"enabled"`, or `"disabled"`.</ParamField>
  </Expandable>
</ParamField>

<ParamField body="cookies" type="array">
  Cookie list injected into the browser context before navigation.

  <Warning>
    `google-trends-explore` ignores omitted, `null`, or empty-array values. Any non-empty or malformed cookies on a Trends request return `400 INVALID_TRENDS_COOKIES`.
  </Warning>

  <Expandable title="cookie fields">
    <ParamField body="name" type="string" required>Cookie name.</ParamField>
    <ParamField body="value" type="string" required>Cookie value.</ParamField>
    <ParamField body="domain" type="string" required>Target domain such as `.example.com`.</ParamField>
    <ParamField body="path" type="string">Cookie path. Defaults to `/`.</ParamField>
    <ParamField body="secure" type="boolean">Whether the cookie is sent only over HTTPS.</ParamField>
    <ParamField body="httpOnly" type="boolean">Whether the cookie is inaccessible to client-side JavaScript.</ParamField>
    <ParamField body="sameSite" type="string">SameSite attribute.</ParamField>
    <ParamField body="session" type="boolean">Set `true` for a session cookie.</ParamField>
    <ParamField body="expirationDate" type="number">Unix expiry timestamp in seconds. Also accepted as `expires` or `expiry`.</ParamField>
  </Expandable>
</ParamField>

## Response

<ResponseField name="mode" type="string">
  `"extract"` or `"inspect"`, mirroring the request mode.
</ResponseField>

<ResponseField name="page" type="object">
  Page metadata.

  <Expandable title="page fields">
    <ResponseField name="url" type="string">
      Final page URL.
    </ResponseField>

    <ResponseField name="title" type="string">
      Page title.
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="data" type="object">
  Extracted structured data. Keys match your `fields` definition or the template's output fields.
</ResponseField>

<ResponseField name="missingFields" type="array">
  List of field keys that were not found.
</ResponseField>

<ResponseField name="source" type="string">
  `"sunbrowser"` — result came from a live browser collection. `"cache"` — result was served from the server cache. Present on Trends responses.
</ResponseField>

<ResponseField name="cached" type="boolean">
  Whether this response was served from cache.
</ResponseField>

<ResponseField name="stale" type="boolean">
  Whether the cached result is stale.
</ResponseField>

<ResponseField name="collectedAt" type="string">
  RFC3339Nano timestamp of when the result was actually collected.
</ResponseField>

<ResponseField name="attempts" type="number">
  Collection attempt count. Cache HIT responses use `0`; stale responses use the attempt count at the time the Worker reported them.
</ResponseField>

<ResponseField name="candidates" type="object">
  Present in `inspect` mode. DOM and network extraction candidates discovered on the page.
</ResponseField>

<ResponseField name="suggestedPlan" type="object">
  Present in `inspect` mode. A plan with `fields` and `schema` you can copy into a future `extract` request.
</ResponseField>

| Code | Meaning |
| - | - |
| 200 | Inspection results or extracted structured data. |
| 400 | Invalid URL, keyword, proxy, region, User-Agent, or Trends cookies. |
| 401 | Missing or invalid `x-api-key`. |
| 402 | Insufficient balance (`INSUFFICIENT_CREDITS`). |
| 404 | Chrome Web Store listing unavailable and CRX fallback could not recover it. |
| 422 | Required fields missing, invalid template or field config, or Worker rejected the task. |
| 429 | Task rate limited, or Google Trends upstream 429. |
| 502 | Proxy or target failure, invalid Trends data, or all candidate identities exhausted. |
| 503 | Queue, Worker, or managed resources unavailable; or Trends identity plan expired. |
| 504 | Task, navigation, or proxy timeout; or Trends response not observed. |
| 500 | Unclassified task execution failure. |

<Note>
  One credit is consumed after request validation but before the task is enqueued. Request bodies are limited to 1 MiB.
</Note>

## More examples

<CodeGroup>
  ```bash Custom fields theme={null}
  curl -sS -X POST "https://api.adscrawl.net/spa-extract" \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -d '{
      "url": "https://example.com/dashboard",
      "mode": "extract",
      "waitUntil": "domcontentloaded",
      "waitFor": { "selector": "h1", "timeoutMs": 15000 },
      "fields": {
        "title":    { "source": "dom", "selector": "h1",           "parse": "string" },
        "subtitle": { "source": "dom", "selector": "p.subtitle",   "parse": "string" },
        "count":    { "source": "dom", "selector": ".metric-value", "parse": "number" }
      },
      "countryCode": "GLOBAL",
      "userAgentMode": "random"
    }'
  ```

  ```bash SimilarWeb template theme={null}
  curl -sS -X POST "https://api.adscrawl.net/spa-extract" \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -d '{
      "template": "similarweb-overview",
      "url": "https://www.similarweb.com/website/dolphin-anty.com/#overview",
      "countryCode": "GLOBAL"
    }'
  ```

  ```bash Google Trends theme={null}
  curl -sS -X POST "https://api.adscrawl.net/spa-extract" \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -d '{
      "template": "google-trends-explore",
      "keyword": "adspower,playwright"
    }'
  ```

  ```bash Inspect mode theme={null}
  curl -sS -X POST "https://api.adscrawl.net/spa-extract" \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -d '{
      "url": "https://example.com/dashboard",
      "mode": "inspect"
    }'
  ```
</CodeGroup>

<RequestExample>
  ```bash cURL theme={null}
  curl -sS -X POST "https://api.adscrawl.net/spa-extract" \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -d '{
      "url": "https://example.com/dashboard",
      "mode": "extract",
      "waitUntil": "domcontentloaded",
      "waitFor": { "selector": "h1", "timeoutMs": 15000 },
      "fields": {
        "title":    { "source": "dom", "selector": "h1",           "parse": "string" },
        "subtitle": { "source": "dom", "selector": "p.subtitle",   "parse": "string" },
        "count":    { "source": "dom", "selector": ".metric-value", "parse": "number" }
      },
      "countryCode": "GLOBAL",
      "userAgentMode": "random"
    }'
  ```

  ```javascript JavaScript theme={null}
  const response = await fetch("https://api.adscrawl.net/spa-extract", {
    method: "POST",
    headers: {
      "content-type": "application/json",
      "x-api-key": "YOUR_API_KEY",
    },
    body: JSON.stringify({
      url: "https://example.com/dashboard",
      mode: "extract",
      waitUntil: "domcontentloaded",
      waitFor: { selector: "h1", timeoutMs: 15000 },
      fields: {
        title:    { source: "dom", selector: "h1",            parse: "string" },
        subtitle: { source: "dom", selector: "p.subtitle",    parse: "string" },
        count:    { source: "dom", selector: ".metric-value", parse: "number" },
      },
      countryCode: "GLOBAL",
      userAgentMode: "random",
    }),
  });

  const result = await response.json();
  console.log(result.data);
  ```

  ```python Python theme={null}
  import httpx

  response = httpx.post(
      "https://api.adscrawl.net/spa-extract",
      headers={
          "content-type": "application/json",
          "x-api-key": "YOUR_API_KEY",
      },
      json={
          "url": "https://example.com/dashboard",
          "mode": "extract",
          "waitUntil": "domcontentloaded",
          "waitFor": {"selector": "h1", "timeoutMs": 15000},
          "fields": {
              "title":    {"source": "dom", "selector": "h1",             "parse": "string"},
              "subtitle": {"source": "dom", "selector": "p.subtitle",     "parse": "string"},
              "count":    {"source": "dom", "selector": ".metric-value",  "parse": "number"},
          },
          "countryCode": "GLOBAL",
          "userAgentMode": "random",
      },
  )

  response.raise_for_status()
  print(response.json()["data"])
  ```
</RequestExample>

<ResponseExample>
  ```json 200 Extract mode theme={null}
  {
    "mode": "extract",
    "page": {
      "url": "https://example.com/dashboard",
      "title": "Example Dashboard"
    },
    "data": {
      "title": "Example Dashboard"
    },
    "missingFields": []
  }
  ```

  ```json 200 Inspect mode theme={null}
  {
    "mode": "inspect",
    "page": {
      "url": "https://example.com/dashboard",
      "title": "Example Dashboard"
    },
    "candidates": {
      "dom": { "metrics": [], "tables": [] },
      "network": []
    },
    "suggestedPlan": {
      "fields": {},
      "schema": {
        "type": "object",
        "properties": {},
        "additionalProperties": false
      }
    }
  }
  ```

  ```json 200 Google Trends theme={null}
  {
    "mode": "extract",
    "page": {
      "url": "https://trends.google.com/trends/explore?q=adspower%2Cplaywright",
      "title": "Google Trends"
    },
    "data": {
      "averages": [
        { "query": "adspower",    "extractedValue": 42 },
        { "query": "playwright",  "extractedValue": 71 }
      ],
      "interestOverTime": []
    },
    "missingFields": [],
    "source": "sunbrowser",
    "cached": false,
    "stale": false,
    "collectedAt": "2026-08-03T02:04:05.123456789Z",
    "attempts": 2
  }
  ```

  ```json 402 Payment required theme={null}
  {
    "code": "INSUFFICIENT_CREDITS"
  }
  ```

  ```json 422 Required fields missing theme={null}
  {
    "code": "SPA_REQUIRED_FIELDS_MISSING"
  }
  ```
</ResponseExample>