# AdsCrawl full documentation

Source: https://docs.adscrawl.net/llms-full.txt
Snapshot: 2026-11-06. Section source URLs identify the canonical documentation.
The current pricing comparison is at https://www.adscrawl.net/pricing/#browser-hour-pricing.

# ChatGPT scraper API
Source: https://docs.adscrawl.net/api-reference/ai-search

POST https://api.adscrawl.net/ai-search
POST /ai-search captures ChatGPT answers and available sources. Configure the prompt, proxy region, browser language and timeout for your analysis.

Use AdsCrawl ChatGPT Scraper to collect an answer and its available source information for brand research, citation analysis or your own GEO workflow. Configure the request in [AI Search Insights](https://app.adscrawl.net/dashboard/ai-search/), then open **API integration** to copy the request for those settings.

## Authentication

Send your AdsCrawl API key in the `x-api-key` header and a JSON request body. The examples read the key from your application's `ADSCRAWL_API_KEY` environment variable. Set that variable before running them and keep your key on the server.

**x-api-key** `string` *required*

Your AdsCrawl API key. Obtain a key from [API keys](https://app.adscrawl.net/dashboard/keys/).

**Content-Type** `string` *required*

Use `application/json`.

## Request body

**platform** `string`

Use `chatgpt` for the currently available ChatGPT scraper.

**query** `string`

The prompt to submit. The console accepts up to 500 characters and preserves line breaks and original wording. Include the language you want the answer to use in this text.

**country\_code** `string`

Proxy-region selection. `GLOBAL` selects a random exit across supported regions. To use a specific supported region, select it in the console and copy its generated API request.

**locale** `string`

Browser-language selection, such as `en-US`. The console's **Automatic (proxy region)** setting omits this field. Browser language does not determine the language of the answer.

**timeoutMs** `number`

Overall capture timeout in milliseconds. The example uses `120000` (120 seconds). The console exposes a 10 to 120 second timeout control; keep your HTTP client's timeout longer than the capture timeout.

## Request examples

These examples use a global proxy exit and an English-language browser. The client waits up to 140 seconds for a capture configured for 120 seconds.

```bash cURL theme={null}
curl --request POST 'https://api.adscrawl.net/ai-search' \
  --header "x-api-key: $ADSCRAWL_API_KEY" \
  --header 'Content-Type: application/json' \
  --max-time 140 \
  --data '{
    "platform": "chatgpt",
    "query": "Recommend a few good-value phones and include reference sources.",
    "country_code": "GLOBAL",
    "locale": "en-US",
    "timeoutMs": 120000
  }'
```

```python Python (requests) theme={null}
import os
import requests

response = requests.post(
    "https://api.adscrawl.net/ai-search",
    headers={"x-api-key": os.environ["ADSCRAWL_API_KEY"]},
    json={
        "platform": "chatgpt",
        "query": "Recommend a few good-value phones and include reference sources.",
        "country_code": "GLOBAL",
        "locale": "en-US",
        "timeoutMs": 120000,
    },
    timeout=140,
)
response.raise_for_status()
print(response.json())
```

```javascript Node.js (fetch) theme={null}
const apiKey = process.env.ADSCRAWL_API_KEY;
if (!apiKey) throw new Error("Set ADSCRAWL_API_KEY before running this example.");

const response = await fetch("https://api.adscrawl.net/ai-search", {
  method: "POST",
  headers: { "x-api-key": apiKey, "Content-Type": "application/json" },
  body: JSON.stringify({
    platform: "chatgpt",
    query: "Recommend a few good-value phones and include reference sources.",
    country_code: "GLOBAL",
    locale: "en-US",
    timeoutMs: 120000,
  }),
  signal: AbortSignal.timeout(140000),
});
if (!response.ok) throw new Error(`Request failed: ${response.status}`);
console.log(await response.json());
```

## Response fields

| Field path | Meaning |
| - | - |
| `result_text` | The captured ChatGPT answer. |
| `cited_sources` | Sources actually cited in the answer, when available. |
| `retrieved_sources` | Sources retrieved during the response, when available. Keep these separate from sources cited in the answer. |
| `ads` | Advertising information observed during capture, when present. |
| `country_code` | The proxy-region context associated with the capture. |
| `locale` | The browser-language context associated with the capture. |
| `capture_context.network` | The observed exit-network context. Use it to check the actual exit when comparing regional results. |
| `task_id` | The capture-task identifier. Save it with your result when investigating a request. |

Answers depend on the prompt and the response available during capture. Sources and ads may be absent. Inspect the returned data before building assumptions about nested source, ad or network fields into your application.

## Regions, languages and analysis

Requests use dynamic residential proxies. A selected region alone does not guarantee access or a particular answer. Preserve the prompt, region, language and observed network context when comparing captures. To request a German answer, for example, explicitly ask for German in `query` even if you set the browser language separately.

Use the captured answer and source data in your own brand-mention, citation or visibility analysis. Repeated collection, scheduling and scoring belong to the workflow you build around the API.

## AI Scrapers availability

All four platforms are selectable in [AI Search Insights](https://app.adscrawl.net/dashboard/ai-search/) and use `POST /ai-search`. Send a separate request for each platform.

| Platform | `platform` value | API guide |
| - | - | - |
| ChatGPT | `chatgpt` | [ChatGPT Scraper](/api-reference/ai-search) |
| Gemini | `gemini` | [Gemini Scraper](/api-reference/gemini-scraper) |
| Google AI Mode | `google-ai-mode` | [Google AI Mode Scraper](/api-reference/google-ai-mode-scraper) |
| Google AI Overview | `google-ai-overview` | [Google AI Overview Scraper](/api-reference/google-ai-overview-scraper) |

Keep the query, requested region and browser language consistent when comparing platforms. Preserve each capture’s date, task ID and observed network context. Availability of a platform does not guarantee an answer, citation or ad for every query.


# Issue Live Control Token
Source: https://docs.adscrawl.net/api-reference/cdp-live-token

POST /cdp/live-token
Issue a single-use, 30-second control token to take live interactive control of an existing CDP session via a dedicated WebSocket connection.

When you need to step in and interact with a running CDP session in real time, use this endpoint to issue a short-lived control token. Pass the returned `controlUrl` directly to a WebSocket constructor to begin live control.

**sessionId** `string` *required*

The active CDP session ID you want to take live control of.

## Response

**ok** `boolean`

`true` when the token was issued successfully.

**controlUrl** `string`

A `wss://` WebSocket URL with the single-use `controlToken` already embedded as a query parameter. Pass this directly to `new WebSocket(...)`; do not replace or modify the token.

**expiresAt** `number`

Unix timestamp in milliseconds when the `controlToken` expires. The token is valid for 30 seconds and can only be used once.

| Status | Meaning |
| - | - |
| 200 | Token issued; `controlUrl` and `expiresAt` are in the response body. |
| 400 | `sessionId` is missing from the request body. |
| 401 / 403 | The API key is invalid or does not own the target session. |
| 404 / 409 / 410 | The session is missing, stopping, or expired. |
| 503 | The session backend is unavailable. |
| 500 | Failed to issue or store the control token. |

> [!WARNING]
>
> The `controlToken` expires after **30 seconds** and can only be consumed **once**. If the WebSocket connection fails or is not opened in time, call `POST /cdp/live-token` again to get a fresh token.

## Open the live WebSocket

Open a live-control WebSocket connection to the target session at `wss://api.adscrawl.net/cdp/live/{sessionId}?controlToken={token}`. The server validates the `controlToken`, connects to the session backend, and then upgrades your client connection. Once connected, CDP messages are proxied bidirectionally and you have full interactive control.

**sessionId** `string` *required*

The session ID embedded in the `controlUrl` returned by `POST /cdp/live-token`.

**controlToken** `string` *required*

The single-use token from `POST /cdp/live-token`. It must be used within 30 seconds of issuance and can only be consumed once.

When the live-control WebSocket closes, the server sends one of the following reason codes:

| Code | Reason | Cause |
| - | - | - |
| 1000 | `cdp_upstream_closed` | The upstream CDP session closed normally. |
| 1000 | `idle_timeout` | The session exceeded the configured `idleTimeoutMs`. |
| 1000 | `max_timeout` | The session exceeded the configured `maxSessionMs`. |
| 1011 | `cdp_upstream_disconnected` | The upstream CDP connection dropped unexpectedly. |
| 1011 | `cdp_upstream_error` | An error occurred on the upstream CDP connection. |

| Status | Meaning |
| - | - |
| 101 | WebSocket upgrade succeeded; live control is now active. |
| 401 | `controlToken` is invalid, expired, already used, or does not match `sessionId`. |
| 404 / 409 / 410 | The session is missing, stopping, or expired. |
| 502 | The session backend could not be reached before the 101 upgrade. |
| 503 | The session backend or control infrastructure is unavailable. |

> [!NOTE]
>
> Live control tokens are designed for short bursts of manual or conditional interaction. For fully automated workflows, drive the session directly through `cdpBaseUrl` with Playwright or Puppeteer instead.

```bash cURL theme={null}
curl -sS -X POST "https://api.adscrawl.net/cdp/live-token" \
  -H "content-type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{"sessionId": "SESSION_ID"}'
```

```javascript JavaScript theme={null}
// Step 1: request a live control token
const token = await fetch("https://api.adscrawl.net/cdp/live-token", {
  method: "POST",
  headers: {
    "content-type": "application/json",
    "x-api-key": "YOUR_API_KEY",
  },
  body: JSON.stringify({ sessionId: "SESSION_ID" }),
}).then((r) => r.json());

// Step 2: open the live control WebSocket using the returned controlUrl
const socket = new WebSocket(token.controlUrl);

socket.addEventListener("open", () => {
  console.log("Live control session connected");
});

socket.addEventListener("close", (event) => {
  console.log("Session closed:", event.code, event.reason);
});
```

```json 200 Token issued theme={null}
{
  "ok": true,
  "controlUrl": "wss://api.adscrawl.net/cdp/live/SESSION_ID?controlToken=<single-use-token>",
  "expiresAt": 1785726630000
}
```


# Create CDP Session
Source: https://docs.adscrawl.net/api-reference/cdp-sessions

POST /cdp/sessions
Launch a dedicated Chromium instance and receive a CDP base URL with an embedded data token for authenticated browser control.

Create a new dedicated CDP session. The server starts a Chromium instance, applies your `browserSettings`, and returns a `cdpBaseUrl` you can pass directly to `playwright.connectOverCDP()` or use for manual CDP discovery.

**idleTimeoutMs** `number`

Milliseconds of inactivity before the session is terminated. Values above the server cap are silently clamped.

**maxSessionMs** `number`

Maximum total lifetime of the session in milliseconds. Values above the server cap are silently clamped.

**browserSettings** `object`

Browser configuration applied when the Chromium instance starts.

**browserSettings fields**

**viewport** `object`

Browser window dimensions: `{ "width": number, "height": number }`. CDP sessions apply these as the `--window-size` launch argument.

**locale** `string`

Browser locale, e.g. `en-US`.

**timezoneId** `string`

IANA timezone identifier, e.g. `Asia/Shanghai`.

**geolocation** `object`

Geographic coordinates: `{ "latitude": number, "longitude": number }`.

**proxy** `object`

Custom proxy configuration. Cannot be combined with `countryCode`.

**proxy fields**

**server** `string`

Full proxy URL such as `http://host:port` or `socks5://host:port`. Do not embed credentials here; provide them via `username` and `password` separately. Cannot be combined with `host`.

**protocol** `string`

Proxy protocol for the split form: `http` or `socks5`.

**host** `string`

Proxy host for the split form.

**port** `number`

Proxy port (1 to 65535) for the split form.

**username** `string`

Proxy username.

**password** `string`

Proxy password.

**countryCode** `string`

Managed proxy region. Use `"GLOBAL"` to select a popular region automatically, or a specific region code for trusted proxies with dynamic fallback. Omitting this field picks a random trusted proxy.

**userAgent** `string`

Override the default User-Agent string.

**userAgentMode** `string`

Set to `"random"` to have the server choose from its User-Agent library. Requests without a User-Agent default to `"random"`.

**userAgentOs** `string`

Operating system used when `userAgentMode` is `"random"`. Accepts `"windows"` (default) or `"macos"`.

**fingerprint** `object`

CDP fingerprint overrides. When omitted, `canvas` and `webGlImage` default to `real`; all other signals are generated from a coherent randomized profile.

**fingerprint fields**

**webRtc** `string`

`"forward"` uses the proxy exit address; `"real"` or `"disabled"` also accepted.

**webGl** `string`

WebGL vendor and renderer metadata: `"random"` or `"real"`.

**webGpu** `string`

`"random"` follows the WebGL GPU selection. Also accepts `"real"` or `"disabled"`. Cannot be `"random"` when `webGlImage` is `"real"`.

**webGlImage** `string`

WebGL image noise: `"random"` or `"real"`.

**canvas** `string`

Canvas noise: `"random"` or `"real"`.

**audioContext** `string`

Audio fingerprint noise: `"random"` or `"real"`.

**clientRects** `string`

Layout measurement noise: `"random"` or `"real"`.

**speechVoices** `string`

OS-matched speech voice list: `"random"` or `"real"`.

**fonts** `string`

OS-matched font list: `"random"` or `"real"`.

**hardware** `string`

CPU thread count and memory (generated as a pair): `"random"` or `"real"`.

**doNotTrack** `string`

Do Not Track preference: `"random"`, `"enabled"`, or `"disabled"`.

**cookies** `array`

Cookies injected into the browser context before the session starts.

**cookie fields**

**name** `string` *required*

Cookie name.

**value** `string` *required*

Cookie value.

**domain** `string` *required*

Target domain, e.g. `.example.com`.

**path** `string`

Cookie path. Defaults to `/`.

**secure** `boolean`

When `true`, the cookie is only sent over HTTPS.

**httpOnly** `boolean`

When `true`, the cookie is inaccessible to client-side JavaScript.

**sameSite** `string`

SameSite attribute value.

**session** `boolean`

Set to `true` for a session cookie.

**expirationDate** `number`

Unix expiry timestamp in seconds. Also accepted as `expires` or `expiry`.

## Response

**sessionId** `string`

Unique identifier for the active session. Use it for all subsequent list, delete, and CDP requests.

**expiresAt** `string`

RFC 3339 timestamp when the session will expire.

**cdpBaseUrl** `string`

Fully formed base URL for this session. Pass it directly to `playwright.connectOverCDP()`. It already contains the data token as a query parameter: do not replace or append your API key.

> [!NOTE]
>
> The `cdpBaseUrl` carries an embedded data token that authorizes CDP access. Never replace that token with your `x-api-key`. Store it securely and treat it with the same care as a credential.

| Status | Meaning |
| - | - |
| `201` | Session created successfully. |
| `400` | Invalid managed region, custom proxy, or random User-Agent parameters. |
| `401` | Missing or invalid `x-api-key`. |
| `429` | The API key has reached its concurrent CDP session limit. |
| `502` | The browser backend rejected the payload, returned an oversized result, or failed to start the browser. |
| `503` | Total capacity exceeded, or the browser backend is unavailable. |
| `504` | Session startup or queue timeout. |
| `500` | Unclassified internal error. |

```bash cURL theme={null}
curl -sS -X POST "https://api.adscrawl.net/cdp/sessions" \
  -H "content-type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{
    "idleTimeoutMs": 600000,
    "maxSessionMs": 3600000,
    "browserSettings": {
      "viewport": { "width": 1440, "height": 900 },
      "countryCode": "GLOBAL",
      "userAgentMode": "random",
      "userAgentOs": "windows"
    }
  }'
```

```json Body JSON theme={null}
{
  "idleTimeoutMs": 600000,
  "maxSessionMs": 3600000,
  "browserSettings": {
    "viewport": { "width": 1440, "height": 900 },
    "countryCode": "GLOBAL",
    "userAgentMode": "random",
    "userAgentOs": "windows"
  }
}
```

```json 201 Created theme={null}
{
  "sessionId": "6c3f7d14-7fe4-4c8e-9f1b-0b6d6f2fa2ef",
  "expiresAt": "2026-04-21T10:30:00.000Z",
  "cdpBaseUrl": "https://api.adscrawl.net/cdp/sessions/6c3f7d14-7fe4-4c8e-9f1b-0b6d6f2fa2ef?token=<data-token>"
}
```

```json 429 Rate Limited theme={null}
{
  "error": "CDP sessions per API key limit reached"
}
```


# Delete CDP Session
Source: https://docs.adscrawl.net/api-reference/cdp-sessions-delete

DELETE /cdp/sessions/{sessionId}
Immediately terminate a CDP session and release its capacity so it no longer counts against your concurrency limit.

Immediately terminate a CDP session and release its capacity.

**sessionId** `string` *required*

The session ID to terminate, as returned by `POST /cdp/sessions` or `GET /cdp/sessions`.

## Response

**ok** `boolean`

Whether the deletion succeeded.

| Status | Meaning |
| - | - |
| `200` | Session deleted successfully. |
| `401` | Missing or invalid `x-api-key`. |
| `403` | The session does not belong to the current API key. |
| `404` | Session ID not found. |
| `409` | The session is already in the process of stopping. |
| `410` | The session has already expired. |
| `503` | The assigned browser backend is unavailable. |

> [!TIP]
>
> Always delete sessions when your automation is complete. Unused sessions count against your concurrency limit until they expire or are explicitly deleted.

```bash cURL theme={null}
curl -sS -X DELETE "https://api.adscrawl.net/cdp/sessions/SESSION_ID" \
  -H "x-api-key: YOUR_API_KEY"
```

```json 200 Deleted theme={null}
{
  "ok": true
}
```

```json 404 Not Found theme={null}
{
  "error": "CDP session not found"
}
```


# List CDP Sessions
Source: https://docs.adscrawl.net/api-reference/cdp-sessions-list

GET /cdp/sessions
Retrieve all active CDP sessions associated with your API key, including session IDs, expiry times, and CDP base URLs.

List all active CDP sessions associated with your API key.

## Response

**ok** `boolean`

Whether the request succeeded.

**data** `array`

Array of active session objects.

**session object fields**

**sessionId** `string`

Unique identifier for the active session.

**expiresAt** `string`

RFC 3339 timestamp when the session will expire.

**cdpBaseUrl** `string`

Fully formed base URL for this session. Pass it directly to `playwright.connectOverCDP()`. It already contains the data token as a query parameter: do not replace or append your API key.

| Status | Meaning |
| - | - |
| `200` | Array of sessions for the current key. |
| `401` | Missing or invalid `x-api-key`. |
| `500` | Failed to read sessions or their data tokens. |

```bash cURL theme={null}
curl -sS "https://api.adscrawl.net/cdp/sessions" \
  -H "x-api-key: YOUR_API_KEY"
```

```json 200 OK theme={null}
{
  "ok": true,
  "data": [
    {
      "sessionId": "6c3f7d14-7fe4-4c8e-9f1b-0b6d6f2fa2ef",
      "expiresAt": "2026-04-21T10:30:00.000Z",
      "cdpBaseUrl": "https://api.adscrawl.net/cdp/sessions/6c3f7d14-7fe4-4c8e-9f1b-0b6d6f2fa2ef?token=<data-token>"
    }
  ]
}
```


# Discover CDP Endpoint
Source: https://docs.adscrawl.net/api-reference/cdp-sessions-version

GET /cdp/sessions/{sessionId}/json/version
Retrieve the Chrome DevTools Protocol discovery JSON for a session, including the pre-rewritten WebSocket debugger URL.

Retrieve the Chrome DevTools Protocol discovery JSON for a session. The response includes a `webSocketDebuggerUrl` pre-rewritten to the public, session-scoped CDP WebSocket path. Use it directly rather than constructing the URL manually.

**sessionId** `string` *required*

Session ID returned by `POST /cdp/sessions` or `GET /cdp/sessions`.

**token** `string` *required*

The data token already embedded in the `cdpBaseUrl` returned when the session was created. Do not substitute your `x-api-key` here.

## Response

**Browser** `string`

Chrome version string, e.g. `Chrome/136.0.0.0`.

**Protocol-Version** `string`

CDP protocol version, e.g. `1.3`.

**User-Agent** `string`

The browser's User-Agent string.

**V8-Version** `string`

V8 engine version.

**WebKit-Version** `string`

WebKit version.

**webSocketDebuggerUrl** `string`

Pre-rewritten WebSocket URL for the browser CDP endpoint. Pass this directly to your CDP client.

| Status | Meaning |
| - | - |
| `200` | Chrome discovery JSON with a rewritten `webSocketDebuggerUrl`. |
| `401` | The data token is missing or invalid. |
| `404` / `409` / `410` | The session is missing, stopping, or expired. |
| `502` | Failed to fetch upstream CDP discovery from the cloud backend. |
| `503` | The session backend is unavailable. |

## Connect over WebSocket

Upgrade to a WebSocket connection to the browser's CDP endpoint. CDP messages are proxied bidirectionally between your client and the Chromium instance running in the cloud. Use the `webSocketDebuggerUrl` returned by the discovery endpoint rather than constructing this URL manually.

**sessionId** `string` *required*

Current CDP session ID.

**browserId** `string` *required*

Browser target ID from the discovery endpoint. Prefer copying the full `webSocketDebuggerUrl` directly instead of assembling this path yourself.

**token** `string` *required*

The same data token used for discovery: embedded in `cdpBaseUrl` and in the `webSocketDebuggerUrl` returned by the discovery endpoint.

| Status | Meaning |
| - | - |
| `101` | WebSocket upgrade succeeded; CDP messages are now proxied bidirectionally. |
| `401` | The data token is missing or invalid. |
| `404` / `409` / `410` | The session is missing, stopping, or expired. |
| `502` | The cloud backend CDP connection failed before the 101 upgrade. |
| `503` | The session backend is unavailable. |

```bash cURL theme={null}
curl -sS "https://api.adscrawl.net/cdp/sessions/SESSION_ID/json/version?token=DATA_TOKEN"
```

```javascript JavaScript theme={null}
// Step 1: fetch the discovery document
const discovery = await fetch(
  "https://api.adscrawl.net/cdp/sessions/SESSION_ID/json/version?token=DATA_TOKEN"
).then((response) => response.json());

// Step 2: open a WebSocket using the pre-built URL
const socket = new WebSocket(discovery.webSocketDebuggerUrl);

socket.addEventListener("open", () => {
  console.log("CDP WebSocket connected");
});
```

```json 200 OK theme={null}
{
  "Browser": "Chrome/136.0.0.0",
  "Protocol-Version": "1.3",
  "User-Agent": "Mozilla/5.0 ...",
  "V8-Version": "13.6.233.8",
  "WebKit-Version": "537.36 (@revision)",
  "webSocketDebuggerUrl": "wss://api.adscrawl.net/cdp/sessions/SESSION_ID/devtools/browser/BROWSER_ID?token=<data-token>"
}
```


# Create Browser Profile
Source: https://docs.adscrawl.net/api-reference/cloud-browsers-create

POST /cloud-browsers
Save a new cloud browser profile with optional viewport, locale, proxy, cookies, and fingerprint settings. No browser session starts and no running quota is consumed.

Save a browser configuration (viewport, locale, timezone, proxy, cookies, and fingerprint) and return a persistent profile `id`. No browser starts and no running quota is consumed. Use the returned `id` to start the browser later with `POST /cloud-browsers/{id}/start`, or to query, update, or delete the profile. To create and start in a single request, use `POST /cloud-browsers/launch` instead.

**remark** `string`

Optional human-readable label for this profile. Maximum 255 Unicode characters after trimming. Useful for identifying profiles in list responses and the dashboard.

**browserSettings** `object`

Optional saved configuration for the browser. Defaults to `{}`. When provided, it must be an object (arrays and scalars are rejected). Saving a proxy here does not replace the required top-level `proxy` field on subsequent API key start requests.

**browserSettings fields**

**viewport** `object`

Browser window dimensions: `{ width: number, height: number }`.

**locale** `string`

Browser locale, such as `en-US`.

**timezoneId** `string`

IANA timezone identifier, such as `Asia/Shanghai`.

**proxy** `object`

Custom proxy configuration saved to the profile. Cannot be combined with `countryCode`. Provide either `server` (for example, `http://host:port`) or the split form `protocol` + `host` + `port`. Credentials must not be embedded in the URL. Supply `username` and `password` as separate fields. Saving a proxy here does not satisfy the requirement to send a top-level `proxy` on API key start requests.

**countryCode** `string`

Managed proxy region saved to the profile. Use `GLOBAL` for a random popular region, or a two-letter ISO country code such as `FR`. Cannot be combined with `proxy`.

**cookies** `array`

Preset cookies stored encrypted with the profile and restored on each run. Must be an array. Up to 10,000 entries within the 1 MiB body limit. Each cookie requires `name`, `value`, and `domain`; `path` defaults to `/`.

**fingerprint** `object`

Browser fingerprint settings saved to the profile. Each field is optional. Supported signals: `webRtc` (`forward` | `real` | `disabled`), `webGl`, `webGlImage`, `canvas`, `audioContext`, `clientRects`, `speechVoices`, `fonts`, `hardware` (all `random` | `real`), `webGpu` (`random` | `real` | `disabled`), `doNotTrack` (`random` | `enabled` | `disabled`).

## Response

**ok** `boolean`

Always `true` on success.

**id** `string (UUID)`

Persistent identifier for the new profile. Store this value; you need it for every subsequent start, stop, query, and delete call.

> [!NOTE]
>
> The Free plan allows one saved profile. Check `limit` from `GET /cloud-browsers` to see your current allowance before creating additional profiles.

### Error status codes

| Code | Meaning |
| - | - |
| 400 | Empty body, `null`, arrays, scalars, invalid field types, oversized `remark`, invalid cookies, `INVALID_PROXY`, `INVALID_COUNTRY_CODE`, or `COUNTRY_PROXY_CONFLICT` (both `proxy` and `countryCode` supplied together). |
| 401 | Authentication is missing, invalid, or expired. |
| 409 | Your saved-profile allowance is full. Delete an existing profile or upgrade your plan before creating a new one. This error is distinct from the running quota error (`CLOUD_BROWSER_CONCURRENCY_LIMIT`). |
| 500 | An internal service failure. |

```bash cURL theme={null}
curl --fail-with-body --silent --show-error --max-time 65 \
  -X POST 'https://api.adscrawl.net/cloud-browsers' \
  -H 'x-api-key: <api-key>' \
  -H 'content-type: application/json' \
  --data '{
  "remark": "work profile",
  "browserSettings": {
    "viewport": {
      "width": 1440,
      "height": 900
    }
  }
}'
```

```javascript JavaScript theme={null}
// Node.js 20+, save as .mjs. Replace proxy placeholders before running.
const baseUrl = "https://api.adscrawl.net";
const apiKey = process.env.ADSCRAWL_API_KEY;
if (!apiKey) throw new Error("ADSCRAWL_API_KEY is required");

const proxy = {
  server: "http://proxy.example.com:8080",
  username: "<proxy-user>",
  password: "<proxy-password>",
};

async function request(method, path, body, timeoutMs = 65000) {
  const res = await fetch(baseUrl + path, {
    method,
    headers: { "x-api-key": apiKey, "content-type": "application/json" },
    body: body === undefined ? undefined : JSON.stringify(body),
    signal: AbortSignal.timeout(timeoutMs),
  });
  const data = await res.json();
  if (!res.ok) {
    const error = new Error(data.error || "HTTP " + res.status);
    error.status = res.status;
    error.code = data.code;
    error.id = data.id;
    throw error;
  }
  return data;
}

async function stopAndWait(id) {
  const path = "/cloud-browsers/" + encodeURIComponent(id);
  for (let attempt = 0; attempt < 30; attempt++) {
    try {
      const stopped = await request("POST", path + "/stop", undefined, 10000);
      if (stopped.runtime.status === "stopped") return;
      const current = await request("GET", path, undefined, 10000);
      if (current.runtime.status === "stopped") return;
    } catch (error) {
      if (
        error.code !== "CDP_SESSION_STARTING" &&
        !(error.status >= 500) &&
        error.name !== "TimeoutError"
      )
        throw error;
    }
    await new Promise((resolve) => setTimeout(resolve, 2000));
  }
  throw new Error(
    "Stop unconfirmed; quota is still reserved. Retry stop for " + id
  );
}

// 1. Save a profile — no browser starts yet.
const { id } = await request("POST", "/cloud-browsers", {
  remark: "work profile",
  browserSettings: { viewport: { width: 1440, height: 900 } },
});

try {
  // 2. Provide a proxy on EVERY start, even for an existing profile.
  await request(
    "POST",
    "/cloud-browsers/" + encodeURIComponent(id) + "/start",
    { proxy }
  );
  const current = await request(
    "GET",
    "/cloud-browsers/" + encodeURIComponent(id)
  );
  console.log({
    id,
    source: current.source,
    status: current.runtime.status,
    connectUrl: current.runtime.connectUrl,
  });
  const { limit, runningLimit, runningCount } = await request(
    "GET",
    "/cloud-browsers"
  );
  console.log({ limit, runningLimit, runningCount });
} finally {
  await stopAndWait(id);
}
console.log("Stopped; the saved profile remains", id);
```

```json 201 Created theme={null}
{
  "ok": true,
  "id": "<browser-id>"
}
```


# Get Cloud Browser Profile
Source: https://docs.adscrawl.net/api-reference/cloud-browsers-get

GET /cloud-browsers/{id}
Fetch the full state of a single saved cloud browser profile by its persistent UUID. Check runtime status, connect URL, and profile settings.

Fetch the full state of a single saved cloud browser profile by its persistent `id`. Use this endpoint to check the current `runtime.status` of a specific profile, retrieve its `connectUrl` when running, or poll during startup and shutdown. This endpoint accepts `x-api-key`, a session JWT in `Authorization: Bearer <token>`, or a dashboard session cookie.

**id** `string (UUID)` *required*

The persistent profile identifier returned by `POST /cloud-browsers` (create), `POST /cloud-browsers/launch`, or `GET /cloud-browsers` (list). This is distinct from the runtime `sessionId`.

## Response

**id** `string (UUID)`

Persistent profile identifier.

**remark** `string | null`

Optional label saved at creation. `null` if not set.

**browserSettings** `object`

Saved configuration (viewport, locale, timezoneId, proxy display info, cookies, fingerprint). Proxy credentials are stripped from responses.

**proxyDisplayIp** `string | null`

Last observed proxy exit IP address, or `null` if not yet set.

**proxyDisplayRegion** `string | null`

Last observed proxy region, or `null` if not yet set.

**lastOpenedAt** `string (RFC3339) | null`

Timestamp of the most recent successful start, or `null` if the profile has never run.

**updatedAt** `string (RFC3339)`

Timestamp of the most recent profile update.

**runtime** `object`

Current runtime status for this profile. See the runtime fields described in the list response.

| Code | Meaning |
| - | - |
| `200` | Success |
| `401` | Unauthenticated |
| `403` | Session limit reached (`SESSIONS_PER_API_KEY_LIMIT_REACHED`) |
| `404` | Not found |
| `500` | Internal error |
| `503` | Runtime unavailable |

```bash cURL theme={null}
curl --fail-with-body --silent --show-error --max-time 65 \
  -X GET 'https://api.adscrawl.net/cloud-browsers/<browser-id>' \
  -H 'x-api-key: <api-key>'
```

```json 200 Stopped theme={null}
{
  "id": "<browser-id>",
  "source": "manual",
  "deleteOnStop": false,
  "remark": "work profile",
  "browserSettings": {
    "viewport": {
      "width": 1440,
      "height": 900
    }
  },
  "proxyDisplayIp": null,
  "proxyDisplayRegion": null,
  "lastOpenedAt": null,
  "updatedAt": "2026-09-07T08:00:00.000Z",
  "runtime": {
    "runtimeKind": "neko",
    "status": "stopped"
  }
}
```

```json 200 Running theme={null}
{
  "id": "<browser-id>",
  "source": "manual",
  "deleteOnStop": false,
  "remark": "work profile",
  "browserSettings": {
    "viewport": {
      "width": 1440,
      "height": 900
    }
  },
  "proxyDisplayIp": "203.0.113.42",
  "proxyDisplayRegion": "US",
  "lastOpenedAt": "2026-09-07T08:30:00.000Z",
  "updatedAt": "2026-09-07T08:00:00.000Z",
  "runtime": {
    "runtimeKind": "neko",
    "status": "running",
    "sessionId": "<session-id>",
    "expiresAt": "2026-09-07T09:30:00.000Z",
    "connectUrl": "https://api.adscrawl.net/cloud-browser-runtime/<session-id>/?usr=adscrawl&pwd=adscrawl"
  }
}
```


# Create and Launch Browser
Source: https://docs.adscrawl.net/api-reference/cloud-browsers-launch

POST /cloud-browsers/launch
Create a cloud browser profile and start it in one request. Returns 201 only after the browser reaches running state with a live connectUrl.

Use this endpoint when you want a running browser immediately without a separate create-then-start flow. A single `POST /cloud-browsers/launch` saves a new profile and starts it, blocking until the browser reaches `running` state before returning `201`. The response includes `runtime.connectUrl`. Open that URL in a browser signed in as the profile owner to access the interactive session. The saved profile persists after you stop the browser and can be restarted, queried, or deleted just like any manually created profile.

> [!WARNING]
>
> Each call to this endpoint creates a new profile with no idempotency key. Never automatically retry a failed launch request. If the response includes an `id`, inspect that profile and call `POST /cloud-browsers/{id}/stop` (with retries) before attempting a new launch.

**proxy** `object` *required*

Custom proxy configuration. Required on every request regardless of authentication method. No saved proxy, managed region, or previous run's proxy can replace this field.

Supply either the `server` form or the split `protocol` + `host` + `port` form:

* `server` — full URL: `http://proxy.example.com:8080` or `socks5://proxy.example.com:1080`. Must include an explicit port (1–65535). Must not contain embedded credentials, path, query, or fragment. Cannot be combined with `host`.
* `protocol` — `http` or `socks5`.
* `host` — proxy hostname.
* `port` — integer or numeric string from 1 to 65535.

Omit both `username` and `password` for an unauthenticated proxy, or supply both as non-empty strings. A proxy failure must never fall back to a direct connection.

**tabs** `array`

HTTP(S) URLs to open as tabs when the browser starts. Defaults to `[]` (no tabs injected). Up to 8 entries; each URL must be at most 16,384 bytes without embedded credentials, control characters, or surrounding whitespace. Each entry is either a URL string or an object `{ url, active? }`. Set `active: true` on at most one tab to make it the active tab; the first tab becomes active when none is selected.

**cookies** `array`

Cookies to inject before the browser opens. Defaults to `[]`. Up to 10,000 entries within the 1 MiB request limit. Required fields per entry: `name`, `domain` (non-empty strings); `value` defaults to an empty string. Optional fields: `path` (defaults to `/`), `secure`, `httpOnly`, `session` (booleans), `expires` (Unix seconds), `sameSite` (`Strict` | `Lax` | `None`). Expired entries are filtered; unknown fields and invalid types are rejected.

**fingerprint** `object`

Browser fingerprint settings for this session. Defaults: `webRtc=forward`; all other signals (`webGl`, `webGpu`, `webGlImage`, `canvas`, `audioContext`, `clientRects`, `speechVoices`, `fonts`, `hardware`, `doNotTrack`) default to random. Partial input fills remaining defaults. Optional `hardwareConcurrency` and `deviceMemory` accept integers 1–64; the server generates the runtime seed.

**apiKeyId** `string (UUID)`

Required when authenticating with a session cookie or `Authorization: Bearer`. Selects which of your active API keys to use for billing. When authenticating with `x-api-key` directly, this field is optional. If supplied, it must match the key used in the header.

## Response

Returned only after the browser reaches `running` state. The `Location` response header points to `GET /cloud-browsers/{id}` for subsequent queries.

**ok** `boolean`

Always `true` on success.

**id** `string (UUID)`

Persistent profile identifier. Store this immediately; you need it to stop, query, or delete the browser.

**source** `string`

Always `"launch"` for profiles created via this endpoint.

**deleteOnStop** `boolean`

Always `false`. The profile is retained after the browser stops.

**runtime** `object`

Runtime state at the moment the response is sent.

**Runtime fields**

**runtimeKind** `string`

`neko` or `worker_cdp`. `neko` omits `cdpBaseUrl`.

**status** `string`

Always `"running"` in a successful `201` response.

**sessionId** `string`

Active session identifier.

**expiresAt** `string (RFC3339)`

Session expiry timestamp.

**connectUrl** `string`

Direct URL to the interactive browser viewer. Open this in a browser signed in as the profile owner. Use the URL exactly as returned; never construct it manually or append API keys, cookies, or proxy credentials.

### Error status codes

| Code | Meaning |
| - | - |
| 400 | `PROXY_REQUIRED`, `INVALID_PROXY`, `INVALID_TABS`, `INVALID_COOKIES`, or `INVALID_FINGERPRINT_SETTINGS`. Explicit `null`, unknown fields, invalid JSON, and incorrect types are all rejected before any profile is created. |
| 401 | Authentication is missing, invalid, or expired. |
| 402 | `PAID_PLAN_REQUIRED` or `INSUFFICIENT_CREDITS`. No profile is created. |
| 403 | The supplied `apiKeyId` belongs to another user or does not match the authenticated key. No profile is created. |
| 409 | Either the saved-profile allowance is full (`Cloud Browser limit reached`) or the running allowance is full or zero (`CLOUD_BROWSER_CONCURRENCY_LIMIT`). When a profile was created before the conflict was detected, the response includes `id`, `source: "launch"`, `deleteOnStop: false`, `deleted: false`, and the actual runtime status. The profile remains saved and counts toward your allowance. |
| 502 / 503 / 504 / 500 | Errors that occur after profile creation include `id`, `runtime`, and `Location` in the response body. Cleanup may leave the session in `stopping` state, which still reserves a running slot until confirmed stopped. |

> [!WARNING]
>
> When the response contains an `id` alongside a 5xx status, do not automatically retry the launch. Instead, call `POST /cloud-browsers/{id}/stop` with a bounded retry loop until `runtime.status` reaches `stopped`, then decide whether to launch again.

```bash cURL theme={null}
curl --fail-with-body --silent --show-error --max-time 200 \
  -X POST 'https://api.adscrawl.net/cloud-browsers/launch' \
  -H 'x-api-key: <api-key>' \
  -H 'content-type: application/json' \
  --data '{
  "proxy": {
    "server": "http://proxy.example.com:8080",
    "username": "<proxy-user>",
    "password": "<proxy-password>"
  },
  "tabs": [
    "https://example.com"
  ],
  "cookies": [
    {
      "name": "sid",
      "value": "<cookie-value>",
      "domain": "example.com",
      "path": "/",
      "secure": true
    }
  ],
  "fingerprint": {
    "canvas": "real"
  }
}'
```

```json 201 Running theme={null}
{
  "ok": true,
  "id": "<browser-id>",
  "source": "launch",
  "deleteOnStop": false,
  "runtime": {
    "runtimeKind": "neko",
    "status": "running",
    "sessionId": "<session-id>",
    "expiresAt": "2026-09-07T09:00:00.000Z",
    "connectUrl": "https://api.adscrawl.net/cloud-browser-runtime/<session-id>/?usr=adscrawl&pwd=adscrawl"
  }
}
```

```json 504 Stopping (cleanup in progress) theme={null}
{
  "error": "Cloud browser runtime request timed out",
  "code": "CLOUD_RUNTIME_TIMEOUT",
  "id": "<browser-id>",
  "source": "launch",
  "deleteOnStop": false,
  "deleted": false,
  "runtime": {
    "runtimeKind": "neko",
    "status": "stopping",
    "sessionId": "<session-id>",
    "expiresAt": "2026-09-07T09:00:00.000Z"
  }
}
```


# List Cloud Browser Profiles
Source: https://docs.adscrawl.net/api-reference/cloud-browsers-list

GET /cloud-browsers
List all saved cloud browser profiles for your account along with pagination metadata and your plan's saved-profile and concurrent-running allowances.

Retrieve every cloud browser profile saved under your account together with account-wide quota fields that tell you how many profiles you can save (`limit`), how many can run at the same time (`runningLimit`), and how many are currently occupying a running slot (`runningCount`). This endpoint accepts `x-api-key`, a session JWT in `Authorization: Bearer <token>`, or a dashboard session cookie.

**page** `integer` (default: 1)

Page number to retrieve. Defaults to `1`; invalid or non-positive values fall back to `1`. Maximum is `10000`.

**pageSize** `integer` (default: 10)

Number of profiles per page. Defaults to `10`; invalid or non-positive values fall back to `10`. Maximum is `10`.

## Response

**ok** `boolean`

Always `true` on success.

**data** `array`

List of saved cloud browser profile objects for the requested page.

**Profile object fields**

**id** `string (UUID)`

Persistent profile identifier. Use this value in path parameters for start, stop, detail, and delete.

**source** `string`

Origin of the profile: `manual` for profiles created with `POST /cloud-browsers`, or `launch` for profiles created with `POST /cloud-browsers/launch`.

**deleteOnStop** `boolean`

Always `false` in current API responses. Profiles are retained after stop unless explicitly deleted.

**remark** `string | null`

Optional label saved at creation or via PATCH.

**browserSettings** `object`

Saved configuration (viewport, locale, timezoneId, proxy display info, cookies, fingerprint). Proxy credentials are stripped from responses.

**proxyDisplayIp** `string | null`

Last observed proxy exit IP address, or `null` if not yet set.

**proxyDisplayRegion** `string | null`

Last observed proxy region, or `null` if not yet set.

**lastOpenedAt** `string (RFC3339) | null`

Timestamp of the most recent successful start, or `null` if the profile has never run.

**updatedAt** `string (RFC3339)`

Timestamp of the most recent profile update.

**runtime** `object`

Current runtime status for this profile.

**Runtime fields**

**runtimeKind** `string`

`neko` or `worker_cdp`.

**status** `string`

`starting`, `running`, `stopping`, or `stopped`. Both `starting` and `stopping` reserve a running slot.

**sessionId** `string`

Present when an active session exists.

**expiresAt** `string (RFC3339)`

Session expiry timestamp; present when an active session exists.

**connectUrl** `string`

Direct URL to the interactive browser viewer. Only present when `status` is `running`. Open it in a browser signed in as the profile owner. Never append API keys or proxy credentials to this URL.

**pagination** `object`

Standard pagination envelope.

**Pagination fields**

**page** `integer`

Current page number.

**pageSize** `integer`

Number of items returned on this page.

**total** `integer`

Total number of saved profiles across all pages.

**totalPages** `integer`

Total number of pages given the current `pageSize`.

**limit** `integer`

Your plan's saved-profile allowance: the maximum number of profiles you can save at once. Creating a new profile when you have reached this limit returns `409`.

**runningLimit** `integer`

Your account's concurrent running allowance. A value of `0` blocks all new starts. An unset or negative limit defaults to `1`. Lowering this value does not stop currently running sessions.

**runningCount** `integer`

Total number of sessions currently in `starting`, `running`, or `stopping` state across all your profiles, pages, and API keys. This count excludes temporary Remote CDP sessions created via `/cdp/sessions`.

| Code | Meaning |
| - | - |
| `200` | Success |
| `401` | Unauthenticated |
| `403` | Session limit reached (`SESSIONS_PER_API_KEY_LIMIT_REACHED`) |
| `500` | Internal error |
| `503` | Runtime unavailable |

> [!NOTE]
>
> `runningCount` includes sessions in **all three active states** (`starting`, `running`, and `stopping`) because all three reserve a running slot. A session does not release its slot until it reaches `stopped`.

```bash cURL theme={null}
curl --fail-with-body --silent --show-error --max-time 65 \
  -X GET 'https://api.adscrawl.net/cloud-browsers?page=1&pageSize=10' \
  -H 'x-api-key: <api-key>'
```

```json 200 OK theme={null}
{
  "ok": true,
  "data": [
    {
      "id": "<browser-id>",
      "source": "manual",
      "deleteOnStop": false,
      "remark": "work profile",
      "browserSettings": {
        "viewport": {
          "width": 1440,
          "height": 900
        }
      },
      "proxyDisplayIp": null,
      "proxyDisplayRegion": null,
      "lastOpenedAt": null,
      "updatedAt": "2026-09-07T08:00:00.000Z",
      "runtime": {
        "runtimeKind": "neko",
        "status": "stopped"
      }
    }
  ],
  "pagination": {
    "page": 1,
    "pageSize": 10,
    "total": 1,
    "totalPages": 1
  },
  "limit": 10,
  "runningLimit": 1,
  "runningCount": 0
}
```


# Start Cloud Browser
Source: https://docs.adscrawl.net/api-reference/cloud-browsers-start

POST /cloud-browsers/{id}/start
Start a saved cloud browser profile and wait for it to reach running state. Returns 200 with a live connectUrl only after the browser is ready.

Start a previously saved cloud browser profile by its `id`. The endpoint blocks until the browser reaches `running` state and then returns `200` with the full runtime object, including `connectUrl`, so you can open the interactive session immediately. A successful `200` response means billing has begun: credits are consumed at one credit per started minute from this moment until you call `POST /cloud-browsers/{id}/stop`.

**id** `string (UUID)` *required*

The persistent profile identifier returned by `POST /cloud-browsers` (create) or `GET /cloud-browsers` (list). This is distinct from the runtime `sessionId`.

**proxy** `object` *required*

Required on every `x-api-key` start. You must supply a valid top-level `proxy` on each start request even if a proxy was saved in the profile's `browserSettings`. A saved proxy, a managed `countryCode`, or a previous run's proxy cannot satisfy this requirement.

Supply either the `server` form or the split `protocol` + `host` + `port` form:

* `server` — full URL: `http://proxy.example.com:8080` or `socks5://proxy.example.com:1080`. Must include an explicit port (1–65535). Must not contain embedded credentials, path, query, or fragment. Cannot be combined with `host`.
* `protocol` — `http` or `socks5`.
* `host` — proxy hostname.
* `port` — integer or numeric string from 1 to 65535.

Omit both `username` and `password` for an unauthenticated proxy, or supply both as non-empty strings. A proxy failure must never fall back to a direct connection. Start overrides affect this run only and do not update the saved profile.

**apiKeyId** `string (UUID)`

Required when authenticating with a session cookie or `Authorization: Bearer`. Select an active, unexpired API key that belongs to your account. When authenticating with `x-api-key`, the current key is selected automatically; if `apiKeyId` is also provided, it must match.

**countryCode** `string`

Available to session/Bearer callers only; not accepted with `x-api-key`. Use `GLOBAL` for a random popular region or a two-letter ISO code such as `FR` to prefer a trusted proxy in that region with dynamic fallback. An empty string is not a valid selection. Cannot be combined with `proxy` in the same request. This override affects this run only and does not update the saved profile.

**cookies** `array`

Optional per-run cookie override. Takes precedence over any cookies saved in the profile for this run only, without updating the saved profile configuration.

**fingerprint** `object`

Optional per-run fingerprint override, merged by field with saved settings for this run only. Supported fields: `webRtc` (`forward` | `real` | `disabled`), `webGl` / `webGlImage` / `canvas` / `audioContext` / `clientRects` / `speechVoices` / `fonts` / `hardware` (`random` | `real`), `webGpu` (`random` | `real` | `disabled`), `doNotTrack` (`random` | `enabled` | `disabled`). Legacy `hardwareConcurrency` and `deviceMemory` accept integers 1–64. Invalid or conflicting combinations return `INVALID_FINGERPRINT_SETTINGS`.

## Response

**ok** `boolean`

Always `true` on success.

**runtime** `object`

Runtime state at confirmation.

**Runtime fields**

**runtimeKind** `string`

`neko` or `worker_cdp`. `neko` omits `cdpBaseUrl`.

**status** `string`

Always `"running"` in a successful `200` response.

**sessionId** `string`

Active session identifier.

**expiresAt** `string (RFC3339)`

Session expiry timestamp.

**connectUrl** `string`

Direct URL to the interactive browser viewer. Open this in a browser signed in as the profile owner. Use the URL exactly as returned.

> [!NOTE]
>
> Billing runs at **1 credit per started minute** from a successful `200` response until you confirm stop via `POST /cloud-browsers/{id}/stop`. Closing the viewer tab or losing the connection does **not** stop billing.

| Status | Meaning |
| - | - |
| 200 | Browser is running. Billing has started. |
| 400 | `PROXY_REQUIRED` (missing top-level proxy for API key), `INVALID_PROXY`, `COUNTRY_PROXY_CONFLICT` (`proxy` and `countryCode` both supplied), `INVALID_COUNTRY_CODE`, `INVALID_FINGERPRINT_SETTINGS`, or invalid cookies. A missing or inactive `apiKeyId` when using session/Bearer auth also returns `400`. |
| 401 | Authentication is missing, invalid, or expired. |
| 402 | `PAID_PLAN_REQUIRED` (no active paid plan) or `INSUFFICIENT_CREDITS` (fewer than one credit available). The response includes `balance` and `requiredCredits` fields. |
| 403 | The supplied `apiKeyId` belongs to another user or does not match the authenticated key. |
| 404 | The profile does not exist or belongs to another user. |
| 409 | `CLOUD_BROWSER_CONCURRENCY_LIMIT`: your running allowance is full or set to `0`. Stop another browser to free a slot before retrying. The response includes `runningLimit` and `runningCount`. Or the browser is already running (no code): this profile already has an active session in `starting`, `running`, or `stopping` state. Stop it first, wait for `stopped`, then start again. |
| 502 | `CLOUD_RUNTIME_AUTH_FAILED`, `CLOUD_RUNTIME_NOT_FOUND`, `CLOUD_RUNTIME_HTTP_ERROR`, `CLOUD_RUNTIME_INVALID_RESPONSE`, or `CLOUD_RUNTIME_CONTROLLER_FAILED`. |
| 503 | `CLUSTER_NO_CAPACITY` (includes nullable `nextAvailableAt` and `retryAfterMs`), `CDP session capacity exhausted`, `MANAGED_PROXY_UNAVAILABLE`, `DYNAMIC_PROXY_NOT_CONFIGURED`, `CLOUD_RUNTIME_UNREACHABLE`, or `CLOUD_RUNTIME_CONFIG_INVALID`. A proxy failure never falls back to a direct connection; fix the proxy before retrying. |
| 504 | `CLOUD_RUNTIME_TIMEOUT`. A timeout does not prove the runtime is absent. Inspect the profile's status and call stop when needed; the running slot remains reserved until cleanup is confirmed. |
| 500 | Internal service failure. |

```bash cURL (API Key with Proxy) theme={null}
curl --fail-with-body --silent --show-error --max-time 65 \
  -X POST 'https://api.adscrawl.net/cloud-browsers/<browser-id>/start' \
  -H 'x-api-key: <api-key>' \
  -H 'content-type: application/json' \
  --data '{
  "proxy": {
    "server": "http://proxy.example.com:8080",
    "username": "<proxy-user>",
    "password": "<proxy-password>"
  }
}'
```

```bash cURL (Session / Bearer with countryCode) theme={null}
curl --fail-with-body --silent --show-error --max-time 65 \
  -X POST 'https://api.adscrawl.net/cloud-browsers/<browser-id>/start' \
  -H 'Authorization: Bearer <SESSION_JWT>' \
  -H 'content-type: application/json' \
  --data '{
  "apiKeyId": "<api-key-id>",
  "countryCode": "GLOBAL"
}'
```

```json 200 Running (neko) theme={null}
{
  "ok": true,
  "runtime": {
    "runtimeKind": "neko",
    "status": "running",
    "sessionId": "<session-id>",
    "expiresAt": "2026-09-07T09:00:00.000Z",
    "connectUrl": "https://api.adscrawl.net/cloud-browser-runtime/<session-id>/?usr=adscrawl&pwd=adscrawl"
  }
}
```

```json 400 Proxy Required theme={null}
{
  "error": "API key starts require an explicit proxy in every request",
  "code": "PROXY_REQUIRED"
}
```

```json 409 Concurrency Limit theme={null}
{
  "error": "Cloud browser running limit reached",
  "code": "CLOUD_BROWSER_CONCURRENCY_LIMIT"
}
```

```json 503 No Capacity theme={null}
{
  "error": "Cloud browser cluster has no available capacity",
  "code": "CLUSTER_NO_CAPACITY",
  "nextAvailableAt": null,
  "retryAfterMs": null
}
```


# Stop Cloud Browser
Source: https://docs.adscrawl.net/api-reference/cloud-browsers-stop

POST /cloud-browsers/{id}/stop
Stop a running cloud browser, end billing, and release the concurrent running slot. The saved profile is retained for future restarts.

Stop a running cloud browser to end billing and free up your concurrent running slot. Call this endpoint when you are done with an interactive session. Closing the viewer tab, losing your connection, or letting the session expire in the UI does **not** stop billing. A confirmed `200` response with `runtime.status: stopped` is the only signal that the browser has stopped and the running slot has been released. The saved profile and all its configuration are retained after stop; the browser can be restarted at any time.

**id** `string (UUID)` *required*

The persistent profile identifier returned by create or list. This is distinct from the runtime `sessionId`.

## Response

**ok** `boolean`

Always `true`.

**runtime** `object`

**Runtime fields**

**status** `string`

Always `"stopped"` in a `200` response.

**sessionId** `string`

The session identifier of the session that was stopped. Omitted when no active session existed.

> [!NOTE]
>
> Calling stop on a browser that is already `stopped` returns `200` with `runtime.status: stopped`. The running slot is not double-released and no additional charge is applied.

| Status | Meaning |
| - | - |
| 200 | Stop is confirmed. The running slot has been released. Calling stop again on an already-stopped browser also returns `200`. |
| 202 | Another stop operation is already in progress. The browser is still stopping and the running slot remains reserved. Poll `GET /cloud-browsers/{id}` with a bounded retry loop until `runtime.status` reaches `stopped`. Retry `POST /cloud-browsers/{id}/stop` periodically if the status stays in `stopping`. |
| 401 | Authentication is missing, invalid, or expired. |
| 404 | The profile does not exist or belongs to another user. |
| 409 | `CDP_SESSION_STARTING`: the browser is still in the `starting` phase. Wait for startup to complete, then retry the stop request. |
| 503 | `CDP_WORKER_UNAVAILABLE`, or the cloud browser runtime is temporarily unreachable. A transient loss of contact does not confirm the browser has stopped; the running slot remains reserved. Inspect the profile status and retry stop once reachability is restored. |
| 500 | A failed or timed-out stop is **not** a confirmed stop. The browser may still be in `stopping` state and continue to reserve the running slot. Inspect the profile with `GET /cloud-browsers/{id}` and retry stop after the service recovers. |

> [!WARNING]
>
> **Closing the viewer tab does not stop the browser.** Always call `POST /cloud-browsers/{id}/stop` to end the session and stop billing. Billing continues at 1 credit per minute until a confirmed stop.

> [!NOTE]
>
> After a confirmed stop, the **saved profile is retained**, not deleted. It continues to count toward your saved-profile allowance and can be restarted at any time. Profiles are only removed by an explicit `DELETE /cloud-browsers/{id}` call. Profiles created via `POST /cloud-browsers/launch` follow the same retention policy: `deleteOnStop` is always `false`.

> [!TIP]
>
> Use a **bounded polling loop** for reliable stop confirmation. Call stop, then if the response is `202`, poll `GET /cloud-browsers/{id}` every 2 seconds (up to about 30 attempts) until `runtime.status` is `stopped`. The Node.js lifecycle example in the create profile docs shows a complete implementation.

```bash cURL theme={null}
curl --fail-with-body --silent --show-error --max-time 65 \
  -X POST 'https://api.adscrawl.net/cloud-browsers/<browser-id>/stop' \
  -H 'x-api-key: <api-key>'
```

```json 200 Stopped theme={null}
{
  "ok": true,
  "runtime": {
    "status": "stopped",
    "sessionId": "<session-id>"
  }
}
```

```json 202 Still Stopping theme={null}
{
  "ok": true,
  "runtime": {
    "status": "stopping",
    "sessionId": "<session-id>"
  }
}
```

```json 200 Launch Profile Retained theme={null}
{
  "ok": true,
  "source": "launch",
  "deleteOnStop": false,
  "runtime": {
    "status": "stopped",
    "sessionId": "<session-id>"
  }
}
```

```json 409 Session Starting theme={null}
{
  "error": "CDP session is starting; retry after startup completes",
  "code": "CDP_SESSION_STARTING"
}
```


# Gemini scraper API
Source: https://docs.adscrawl.net/api-reference/gemini-scraper

POST https://api.adscrawl.net/ai-search
POST /ai-search captures Gemini answers and available sources. Configure the prompt, proxy region, browser language and timeout for your analysis.

Use AdsCrawl Gemini Scraper to collect an answer and its available source information for brand research, citation analysis or your own GEO workflow. Configure the request in [AI Search Insights](https://app.adscrawl.net/dashboard/ai-search/), then open **API integration** to copy the request for those settings.

## Authentication

Send your AdsCrawl API key in the `x-api-key` header and a JSON request body. The examples read the key from your application's `ADSCRAWL_API_KEY` environment variable. Set that variable before running them and keep your key on the server.

**x-api-key** `string` *required*

Your AdsCrawl API key. Obtain a key from [API keys](https://app.adscrawl.net/dashboard/keys/).

**Content-Type** `string` *required*

Use `application/json`.

## Request body

**platform** `string`

Use `gemini` to select Gemini. This uses the Gemini capture in AI Search Insights, separately from Google AI Mode and Google AI Overview.

**query** `string`

The prompt to submit. The console accepts up to 500 characters and preserves line breaks and original wording. Include the language you want the answer to use in this text.

**country\_code** `string`

Proxy-region selection. `GLOBAL` selects a random exit across supported regions. To use a specific supported region, select it in the console and copy its generated API request.

**locale** `string`

Browser-language selection, such as `en-US`. The console's **Automatic (proxy region)** setting omits this field. Browser language does not determine the language of the answer.

**timeoutMs** `number`

Overall capture timeout in milliseconds. The example uses `120000` (120 seconds). The console exposes a 10 to 120 second timeout control; keep your HTTP client's timeout longer than the capture timeout.

## Request examples

These examples use a global proxy exit and an English-language browser. The client waits up to 140 seconds for a capture configured for 120 seconds.

```bash cURL theme={null}
curl --request POST 'https://api.adscrawl.net/ai-search' \
  --header "x-api-key: $ADSCRAWL_API_KEY" \
  --header 'Content-Type: application/json' \
  --max-time 140 \
  --data '{
    "platform": "gemini",
    "query": "Recommend a few good-value phones and include reference sources.",
    "country_code": "GLOBAL",
    "locale": "en-US",
    "timeoutMs": 120000
  }'
```

```python Python (requests) theme={null}
import os
import requests

response = requests.post(
    "https://api.adscrawl.net/ai-search",
    headers={"x-api-key": os.environ["ADSCRAWL_API_KEY"]},
    json={
        "platform": "gemini",
        "query": "Recommend a few good-value phones and include reference sources.",
        "country_code": "GLOBAL",
        "locale": "en-US",
        "timeoutMs": 120000,
    },
    timeout=140,
)
response.raise_for_status()
print(response.json())
```

```javascript Node.js (fetch) theme={null}
const apiKey = process.env.ADSCRAWL_API_KEY;
if (!apiKey) throw new Error("Set ADSCRAWL_API_KEY before running this example.");

const response = await fetch("https://api.adscrawl.net/ai-search", {
  method: "POST",
  headers: { "x-api-key": apiKey, "Content-Type": "application/json" },
  body: JSON.stringify({
    platform: "gemini",
    query: "Recommend a few good-value phones and include reference sources.",
    country_code: "GLOBAL",
    locale: "en-US",
    timeoutMs: 120000,
  }),
  signal: AbortSignal.timeout(140000),
});
if (!response.ok) throw new Error(`Request failed: ${response.status}`);
console.log(await response.json());
```

## Response fields

| Field path | Meaning |
| - | - |
| `result_text` | The captured Gemini answer. |
| `cited_sources` | Sources actually cited in the answer, when available. |
| `retrieved_sources` | Sources retrieved during the response, when available. Keep these separate from sources cited in the answer. |
| `ads` | Advertising information observed during capture, when present. |
| `country_code` | The proxy-region context associated with the capture. |
| `locale` | The browser-language context associated with the capture. |
| `capture_context.network` | The observed exit-network context. Use it to check the actual exit when comparing regional results. |
| `task_id` | The capture-task identifier. Save it with your result when investigating a request. |

Answers depend on the prompt and the response available during capture. Sources and ads may be absent. Inspect the returned data before building assumptions about nested source, ad or network fields into your application.

## Regions, languages and analysis

Requests use dynamic residential proxies. A selected region alone does not guarantee access or a particular answer. Preserve the prompt, region, language and observed network context when comparing captures. To request a German answer, for example, explicitly ask for German in `query` even if you set the browser language separately.

Use the captured answer and source data in your own brand-mention, citation or visibility analysis. Repeated collection, scheduling and scoring belong to the workflow you build around the API.

## AI Scrapers availability

All four platforms are selectable in [AI Search Insights](https://app.adscrawl.net/dashboard/ai-search/) and use `POST /ai-search`. Send a separate request for each platform.

| Platform | `platform` value | API guide |
| - | - | - |
| ChatGPT | `chatgpt` | [ChatGPT Scraper](/api-reference/ai-search) |
| Gemini | `gemini` | [Gemini Scraper](/api-reference/gemini-scraper) |
| Google AI Mode | `google-ai-mode` | [Google AI Mode Scraper](/api-reference/google-ai-mode-scraper) |
| Google AI Overview | `google-ai-overview` | [Google AI Overview Scraper](/api-reference/google-ai-overview-scraper) |

Keep the query, requested region and browser language consistent when comparing platforms. Preserve each capture’s date, task ID and observed network context. Availability of a platform does not guarantee an answer, citation or ad for every query.


# Google AI Mode scraper API
Source: https://docs.adscrawl.net/api-reference/google-ai-mode-scraper

POST https://api.adscrawl.net/ai-search
POST /ai-search captures Google AI Mode answers and available sources. Configure the prompt, proxy region, browser language and timeout for your analysis.

Use AdsCrawl Google AI Mode Scraper to collect an answer and its available source information for brand research, citation analysis or your own GEO workflow. Configure the request in [AI Search Insights](https://app.adscrawl.net/dashboard/ai-search/), then open **API integration** to copy the request for those settings.

## Authentication

Send your AdsCrawl API key in the `x-api-key` header and a JSON request body. The examples read the key from your application's `ADSCRAWL_API_KEY` environment variable. Set that variable before running them and keep your key on the server.

**x-api-key** `string` *required*

Your AdsCrawl API key. Obtain a key from [API keys](https://app.adscrawl.net/dashboard/keys/).

**Content-Type** `string` *required*

Use `application/json`.

## Request body

**platform** `string`

Use `google-ai-mode` to select Google AI Mode. Choose `google-ai-overview` in a separate request when you want the Google AI Overview capture.

**query** `string`

The prompt to submit. The console accepts up to 500 characters and preserves line breaks and original wording. Include the language you want the answer to use in this text.

**country\_code** `string`

Proxy-region selection. `GLOBAL` selects a random exit across supported regions. To use a specific supported region, select it in the console and copy its generated API request.

**locale** `string`

Browser-language selection, such as `en-US`. The console's **Automatic (proxy region)** setting omits this field. Browser language does not determine the language of the answer.

**timeoutMs** `number`

Overall capture timeout in milliseconds. The example uses `120000` (120 seconds). The console exposes a 10 to 120 second timeout control; keep your HTTP client's timeout longer than the capture timeout.

## Request examples

These examples use a global proxy exit and an English-language browser. The client waits up to 140 seconds for a capture configured for 120 seconds.

```bash cURL theme={null}
curl --request POST 'https://api.adscrawl.net/ai-search' \
  --header "x-api-key: $ADSCRAWL_API_KEY" \
  --header 'Content-Type: application/json' \
  --max-time 140 \
  --data '{
    "platform": "google-ai-mode",
    "query": "Recommend a few good-value phones and include reference sources.",
    "country_code": "GLOBAL",
    "locale": "en-US",
    "timeoutMs": 120000
  }'
```

```python Python (requests) theme={null}
import os
import requests

response = requests.post(
    "https://api.adscrawl.net/ai-search",
    headers={"x-api-key": os.environ["ADSCRAWL_API_KEY"]},
    json={
        "platform": "google-ai-mode",
        "query": "Recommend a few good-value phones and include reference sources.",
        "country_code": "GLOBAL",
        "locale": "en-US",
        "timeoutMs": 120000,
    },
    timeout=140,
)
response.raise_for_status()
print(response.json())
```

```javascript Node.js (fetch) theme={null}
const apiKey = process.env.ADSCRAWL_API_KEY;
if (!apiKey) throw new Error("Set ADSCRAWL_API_KEY before running this example.");

const response = await fetch("https://api.adscrawl.net/ai-search", {
  method: "POST",
  headers: { "x-api-key": apiKey, "Content-Type": "application/json" },
  body: JSON.stringify({
    platform: "google-ai-mode",
    query: "Recommend a few good-value phones and include reference sources.",
    country_code: "GLOBAL",
    locale: "en-US",
    timeoutMs: 120000,
  }),
  signal: AbortSignal.timeout(140000),
});
if (!response.ok) throw new Error(`Request failed: ${response.status}`);
console.log(await response.json());
```

## Response fields

| Field path | Meaning |
| - | - |
| `result_text` | The captured Google AI Mode answer. |
| `cited_sources` | Sources actually cited in the answer, when available. |
| `retrieved_sources` | Sources retrieved during the response, when available. Keep these separate from sources cited in the answer. |
| `ads` | Advertising information observed during capture, when present. |
| `country_code` | The proxy-region context associated with the capture. |
| `locale` | The browser-language context associated with the capture. |
| `capture_context.network` | The observed exit-network context. Use it to check the actual exit when comparing regional results. |
| `task_id` | The capture-task identifier. Save it with your result when investigating a request. |

Answers depend on the prompt and the response available during capture. Sources and ads may be absent. Inspect the returned data before building assumptions about nested source, ad or network fields into your application.

## Regions, languages and analysis

Requests use dynamic residential proxies. A selected region alone does not guarantee access or a particular answer. Preserve the prompt, region, language and observed network context when comparing captures. To request a German answer, for example, explicitly ask for German in `query` even if you set the browser language separately.

Use the captured answer and source data in your own brand-mention, citation or visibility analysis. Repeated collection, scheduling and scoring belong to the workflow you build around the API.

## AI Scrapers availability

All four platforms are selectable in [AI Search Insights](https://app.adscrawl.net/dashboard/ai-search/) and use `POST /ai-search`. Send a separate request for each platform.

| Platform | `platform` value | API guide |
| - | - | - |
| ChatGPT | `chatgpt` | [ChatGPT Scraper](/api-reference/ai-search) |
| Gemini | `gemini` | [Gemini Scraper](/api-reference/gemini-scraper) |
| Google AI Mode | `google-ai-mode` | [Google AI Mode Scraper](/api-reference/google-ai-mode-scraper) |
| Google AI Overview | `google-ai-overview` | [Google AI Overview Scraper](/api-reference/google-ai-overview-scraper) |

Keep the query, requested region and browser language consistent when comparing platforms. Preserve each capture’s date, task ID and observed network context. Availability of a platform does not guarantee an answer, citation or ad for every query.


# Google AI Overview scraper API
Source: https://docs.adscrawl.net/api-reference/google-ai-overview-scraper

POST https://api.adscrawl.net/ai-search
POST /ai-search captures Google AI Overview answers and available sources. Configure the prompt, proxy region, browser language and timeout for your analysis.

Use AdsCrawl Google AI Overview Scraper to collect an answer and its available source information for brand research, citation analysis or your own GEO workflow. Configure the request in [AI Search Insights](https://app.adscrawl.net/dashboard/ai-search/), then open **API integration** to copy the request for those settings.

## Authentication

Send your AdsCrawl API key in the `x-api-key` header and a JSON request body. The examples read the key from your application's `ADSCRAWL_API_KEY` environment variable. Set that variable before running them and keep your key on the server.

**x-api-key** `string` *required*

Your AdsCrawl API key. Obtain a key from [API keys](https://app.adscrawl.net/dashboard/keys/).

**Content-Type** `string` *required*

Use `application/json`.

## Request body

**platform** `string`

Use `google-ai-overview` to select Google AI Overview. This is a separate capture target from Gemini and Google AI Mode.

**query** `string`

The prompt to submit. The console accepts up to 500 characters and preserves line breaks and original wording. Include the language you want the answer to use in this text.

**country\_code** `string`

Proxy-region selection. `GLOBAL` selects a random exit across supported regions. To use a specific supported region, select it in the console and copy its generated API request.

**locale** `string`

Browser-language selection, such as `en-US`. The console's **Automatic (proxy region)** setting omits this field. Browser language does not determine the language of the answer.

**timeoutMs** `number`

Overall capture timeout in milliseconds. The example uses `120000` (120 seconds). The console exposes a 10 to 120 second timeout control; keep your HTTP client's timeout longer than the capture timeout.

## Request examples

These examples use a global proxy exit and an English-language browser. The client waits up to 140 seconds for a capture configured for 120 seconds.

```bash cURL theme={null}
curl --request POST 'https://api.adscrawl.net/ai-search' \
  --header "x-api-key: $ADSCRAWL_API_KEY" \
  --header 'Content-Type: application/json' \
  --max-time 140 \
  --data '{
    "platform": "google-ai-overview",
    "query": "Recommend a few good-value phones and include reference sources.",
    "country_code": "GLOBAL",
    "locale": "en-US",
    "timeoutMs": 120000
  }'
```

```python Python (requests) theme={null}
import os
import requests

response = requests.post(
    "https://api.adscrawl.net/ai-search",
    headers={"x-api-key": os.environ["ADSCRAWL_API_KEY"]},
    json={
        "platform": "google-ai-overview",
        "query": "Recommend a few good-value phones and include reference sources.",
        "country_code": "GLOBAL",
        "locale": "en-US",
        "timeoutMs": 120000,
    },
    timeout=140,
)
response.raise_for_status()
print(response.json())
```

```javascript Node.js (fetch) theme={null}
const apiKey = process.env.ADSCRAWL_API_KEY;
if (!apiKey) throw new Error("Set ADSCRAWL_API_KEY before running this example.");

const response = await fetch("https://api.adscrawl.net/ai-search", {
  method: "POST",
  headers: { "x-api-key": apiKey, "Content-Type": "application/json" },
  body: JSON.stringify({
    platform: "google-ai-overview",
    query: "Recommend a few good-value phones and include reference sources.",
    country_code: "GLOBAL",
    locale: "en-US",
    timeoutMs: 120000,
  }),
  signal: AbortSignal.timeout(140000),
});
if (!response.ok) throw new Error(`Request failed: ${response.status}`);
console.log(await response.json());
```

## Response fields

| Field path | Meaning |
| - | - |
| `result_text` | The captured Google AI Overview answer. |
| `cited_sources` | Sources actually cited in the answer, when available. |
| `retrieved_sources` | Sources retrieved during the response, when available. Keep these separate from sources cited in the answer. |
| `ads` | Advertising information observed during capture, when present. |
| `country_code` | The proxy-region context associated with the capture. |
| `locale` | The browser-language context associated with the capture. |
| `capture_context.network` | The observed exit-network context. Use it to check the actual exit when comparing regional results. |
| `task_id` | The capture-task identifier. Save it with your result when investigating a request. |

The capture collects the Google AI Overview available for the query. Do not assume every query produces an overview. Answers depend on the prompt and the response available during capture. Sources and ads may be absent. Inspect the returned data before building assumptions about nested source, ad or network fields into your application.

## Regions, languages and analysis

Requests use dynamic residential proxies. A selected region alone does not guarantee access or a particular answer. Preserve the prompt, region, language and observed network context when comparing captures. To request a German answer, for example, explicitly ask for German in `query` even if you set the browser language separately.

Use the captured answer and source data in your own brand-mention, citation or visibility analysis. Repeated collection, scheduling and scoring belong to the workflow you build around the API.

## AI Scrapers availability

All four platforms are selectable in [AI Search Insights](https://app.adscrawl.net/dashboard/ai-search/) and use `POST /ai-search`. Send a separate request for each platform.

| Platform | `platform` value | API guide |
| - | - | - |
| ChatGPT | `chatgpt` | [ChatGPT Scraper](/api-reference/ai-search) |
| Gemini | `gemini` | [Gemini Scraper](/api-reference/gemini-scraper) |
| Google AI Mode | `google-ai-mode` | [Google AI Mode Scraper](/api-reference/google-ai-mode-scraper) |
| Google AI Overview | `google-ai-overview` | [Google AI Overview Scraper](/api-reference/google-ai-overview-scraper) |

Keep the query, requested region and browser language consistent when comparing platforms. Preserve each capture’s date, task ID and observed network context. Availability of a platform does not guarantee an answer, citation or ad for every query.


# Render HTML, Markdown, or Article JSON
Source: https://docs.adscrawl.net/api-reference/html

POST /html
Fetch a fully rendered page in raw HTML, readable Markdown, or structured article JSON, routed through residential proxies.

Use `POST /html` to load any public URL in a real browser and retrieve the rendered output in the format your application needs. Choose `"html"` to get the full DOM, `"markdown"` to get a clean readable article, or `"json"` to get a structured Readability payload with title, author, excerpt, and body content. The request is routed through residential proxies with randomized browser fingerprints.

**url** `string` *required*

Target page URL. Only ports 80 and 443 are supported.

**contentMode** `string`

Controls the response format. Accepted values: `"html"` (default), `"markdown"`, `"json"`. Both `"markdown"` and `"json"` extract readable article content using Readability.

**selector** `string`

Wait for the first matching CSS element before extracting. When `contentMode` is `"html"`, only that element's HTML is returned. When `"markdown"` or `"json"`, Readability runs against that element. Returns `422 CONTENT_SELECTOR_NOT_FOUND` if the selector is not found.

**waitUntil** `string`

Navigation event to wait for before extracting content. Defaults to `"load"`.

| Value | Behaviour |
| - | - |
| `"domcontentloaded"` | Waits for DOMContentLoaded. HTML is parsed without waiting for secondary resources such as images. Recommended for HTML extraction. |
| `"load"` | Waits for `window.load` after the page and dependent resources (images, stylesheets) finish loading. |
| `"networkidle"` | Waits until there are no network connections for at least 500 ms. Long polling, analytics, or lazy-loaded resources may cause a timeout. |

**timeoutMs** `number`

Maximum time to wait for the task to complete, in milliseconds. Must be a positive integer no greater than `3,600,000`. Values outside this range fall back to the server default.

**viewport** `object`

Viewport dimensions used when rendering the page.

**viewport fields**

**width** `number`

Viewport width in pixels.

**height** `number`

Viewport height in pixels.

**locale** `string`

Browser locale, for example `"en-US"` or `"zh-CN"`. Affects `navigator.language` and `Accept-Language` headers.

**timezoneId** `string`

IANA timezone identifier, for example `"Asia/Shanghai"` or `"America/New_York"`.

**geolocation** `object`

Geolocation coordinates exposed to the page via the Geolocation API.

**geolocation fields**

**latitude** `number`

Latitude in decimal degrees.

**longitude** `number`

Longitude in decimal degrees.

**proxy** `object`

Custom proxy configuration. Cannot be combined with `countryCode`. Provide either `server` or the split form (`protocol` + `host` + `port`). Credentials must not be embedded in `server`.

**proxy fields**

**server** `string`

Full proxy URL such as `http://host:port` or `socks5://host:port`. Cannot be combined with `host`.

**protocol** `string`

`"http"` or `"socks5"`. Used in the split form.

**host** `string`

Proxy host. Used in the split form.

**port** `number | string`

Port from 1 to 65535.

**username** `string`

Proxy username. Must be supplied together with `password`.

**password** `string`

Proxy password. Must be supplied together with `username`.

**countryCode** `string`

Managed residential proxy region. Cannot be combined with `proxy`.

* `"GLOBAL"`: dynamic exit from 15 popular regions.
* Two-letter country code (e.g. `"US"`, `"DE"`): prefers a trusted proxy for that region with dynamic fallback.
* Omitted: a random trusted proxy is selected automatically.

**userAgentMode** `string`

`"random"` lets the server pick a User-Agent from its library. Requests without an explicit `userAgent` already default to `"random"`.

**userAgentOs** `string`

Operating system used when `userAgentMode` is `"random"`. Accepted values: `"windows"` (default) or `"macos"`.

**userAgent** `string`

Explicit User-Agent string. Overrides the random selection.

**fingerprint** `object`

Browser fingerprint settings. When omitted, every signal defaults to random while keeping OS, GPU, CPU, memory, fonts, and device signals coherent.

**fingerprint fields**

**webRtc** `string`

`"forward"` uses the proxy exit address. `"real"` or `"disabled"` also accepted.

**webGl** `string`

`"random"` or `"real"`. Controls WebGL vendor and renderer metadata.

**webGpu** `string`

`"random"`, `"real"`, or `"disabled"`. Random mode follows the WebGL GPU setting.

**webGlImage** `string`

`"random"` or `"real"`. Controls WebGL image noise.

**canvas** `string`

`"random"` or `"real"`. Controls canvas noise.

**audioContext** `string`

`"random"` or `"real"`. Controls audio fingerprint noise.

**clientRects** `string`

`"random"` or `"real"`. Controls layout measurement noise.

**speechVoices** `string`

`"random"` or `"real"`. Returns an OS-matched speech voice list.

**fonts** `string`

`"random"` or `"real"`. Returns an OS-matched font list.

**hardware** `string`

`"random"` or `"real"`. Generates a coherent CPU thread count and memory pair.

**doNotTrack** `string`

`"random"`, `"enabled"`, or `"disabled"`.

**cookies** `array`

Cookie list injected into the browser context before navigation. Each cookie object requires `name`, `value`, and `domain`.

**cookie fields**

**name** `string` *required*

Cookie name.

**value** `string` *required*

Cookie value.

**domain** `string` *required*

Target domain such as `.example.com`.

**path** `string`

Cookie path. Defaults to `/`.

**secure** `boolean`

Whether the cookie is sent only over HTTPS.

**httpOnly** `boolean`

Whether the cookie is inaccessible to client-side JavaScript.

**sameSite** `string`

SameSite attribute.

**session** `boolean`

Set `true` for a session cookie.

**expirationDate** `number`

Unix expiry timestamp in seconds. Also accepted as `expires` or `expiry`.

## Response

**title** `string`

Article title extracted from the page.

**byline** `string`

Author name or byline, if present.

**excerpt** `string`

A short summary or description of the article.

**siteName** `string`

Name of the site, if available.

**lang** `string`

Language code of the article, e.g. `"en"`.

**dir** `string`

Text direction, e.g. `"ltr"` or `"rtl"`. `null` if not detected.

**content** `string`

Cleaned HTML of the article body.

**textContent** `string`

Plain-text version of the article body with whitespace normalised.

**length** `number`

Character count of `textContent`.

**publishedTime** `string`

ISO 8601 publication timestamp, or `null` if not found.

| Code | Meaning |
| - | - |
| `200` | Success. Returns text/html, text/markdown, or application/json depending on contentMode. |
| `400` | Invalid JSON, URL, contentMode, cookies, proxy, region, or User-Agent. Body over 1 MiB. |
| `401` | Missing or invalid `x-api-key`. |
| `402` | Insufficient balance. Returns `INSUFFICIENT_CREDITS`, `balance`, and `requiredCredits`. |
| `422` | Selector not found (`CONTENT_SELECTOR_NOT_FOUND`) or readability missing (`READABILITY_CONTENT_NOT_FOUND`). |
| `429` | Task rate limited. |
| `502` | Proxy unreachable or target HTTP failure. |
| `503` | Queue, Worker, managed proxy, or User-Agent resources unavailable. |
| `504` | Task, navigation, or proxy connection timed out. |
| `500` | Unclassified task execution failure. |

> [!NOTE]
>
> One credit is consumed after request validation but before the task is enqueued. Request bodies are limited to 1 MiB.

## More examples

```bash cURL theme={null}
curl -sS -X POST "https://api.adscrawl.net/html" \
  -H "content-type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{
    "url": "https://example.com",
    "contentMode": "html",
    "selector": "main",
    "waitUntil": "domcontentloaded",
    "countryCode": "US"
  }'
```

```javascript JavaScript theme={null}
const response = await fetch("https://api.adscrawl.net/html", {
  method: "POST",
  headers: {
    "content-type": "application/json",
    "x-api-key": "YOUR_API_KEY",
  },
  body: JSON.stringify({
    url: "https://example.com",
    contentMode: "html",
    selector: "main",
    waitUntil: "domcontentloaded",
    countryCode: "US",
  }),
});

const html = await response.text();
console.log(html);
```

```python Python theme={null}
import httpx

response = httpx.post(
    "https://api.adscrawl.net/html",
    headers={
        "content-type": "application/json",
        "x-api-key": "YOUR_API_KEY",
    },
    json={
        "url": "https://example.com",
        "contentMode": "html",
        "selector": "main",
        "waitUntil": "domcontentloaded",
        "countryCode": "US",
    },
)

response.raise_for_status()
print(response.text)
```

```bash cURL theme={null}
curl -sS -X POST "https://api.adscrawl.net/html" \
  -H "content-type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{
    "url": "https://example.com/article",
    "contentMode": "json",
    "waitUntil": "domcontentloaded",
    "viewport": { "width": 1280, "height": 720 },
    "locale": "en-US",
    "countryCode": "GLOBAL",
    "userAgentMode": "random",
    "userAgentOs": "windows"
  }'
```

```javascript JavaScript theme={null}
const response = await fetch("https://api.adscrawl.net/html", {
  method: "POST",
  headers: {
    "content-type": "application/json",
    "x-api-key": "YOUR_API_KEY",
  },
  body: JSON.stringify({
    url: "https://example.com/article",
    contentMode: "json",
    waitUntil: "domcontentloaded",
    viewport: { width: 1280, height: 720 },
    locale: "en-US",
    countryCode: "GLOBAL",
    userAgentMode: "random",
    userAgentOs: "windows",
  }),
});

if (!response.ok) {
  const err = await response.json();
  throw new Error(`${response.status} ${err.code}: ${err.error}`);
}

const article = await response.json();
console.log(article.title, article.byline);
```

```python Python theme={null}
import httpx, json

response = httpx.post(
    "https://api.adscrawl.net/html",
    headers={
        "content-type": "application/json",
        "x-api-key": "YOUR_API_KEY",
    },
    json={
        "url": "https://example.com/article",
        "contentMode": "json",
        "waitUntil": "domcontentloaded",
        "viewport": {"width": 1280, "height": 720},
        "locale": "en-US",
        "countryCode": "GLOBAL",
        "userAgentMode": "random",
        "userAgentOs": "windows",
    },
)

response.raise_for_status()
article = response.json()
print(article["title"], article["byline"])
```

```json 200 JSON (contentMode: json) theme={null}
{
  "title": "Example Article",
  "byline": "OpenAI",
  "excerpt": "A concise article summary.",
  "siteName": "Example",
  "lang": "en",
  "dir": null,
  "content": "<div><p>Readable body...</p></div>",
  "textContent": "Readable body...",
  "length": 2487,
  "publishedTime": null
}
```

```markdown 200 Markdown (contentMode: markdown) theme={null}
# Example Article

Readable body...

- key point one
- key point two
```

```json 402 Insufficient credits theme={null}
{
  "error": "Insufficient credits",
  "code": "INSUFFICIENT_CREDITS",
  "balance": 0,
  "requiredCredits": 1
}
```

```json 422 Readability not found theme={null}
{
  "error": "Readable article content was not found",
  "code": "READABILITY_CONTENT_NOT_FOUND"
}
```

```json 400 Bad country code theme={null}
{
  "error": "countryCode is not supported",
  "code": "INVALID_COUNTRY_CODE"
}
```

```json 503 Proxy unavailable theme={null}
{
  "error": "Dynamic country/region routing is unavailable",
  "code": "DYNAMIC_PROXY_NOT_CONFIGURED"
}
```


# AdsCrawl REST API Introduction
Source: https://docs.adscrawl.net/api-reference/introduction

Base URL, authentication, JSON request bodies, credits consumption, common HTTP status codes, and links to all AdsCrawl API endpoint groups and schemas.

The AdsCrawl REST API lets you render web pages, capture screenshots, extract SPA data, manage remote CDP sessions, and launch persistent cloud browsers programmatically. This page covers the base URL, how to authenticate, how credits are billed, common status codes, and where to find every endpoint group.

## Base URL

All endpoints are served over HTTPS:

```text theme={null}
https://api.adscrawl.net
```

## Authentication

Nearly every endpoint requires an API key in the `x-api-key` request header. Generate and manage keys from the [AdsCrawl dashboard](https://app.adscrawl.net/dashboard/). Store your key in an environment variable or secrets manager, and never expose it in client-side code or public repositories.

Cloud browser lifecycle endpoints (`/cloud-browsers/*`) also accept a session JWT via `Authorization: Bearer <SESSION_JWT>`, or a dashboard session cookie.

All requests with a JSON body must include:

```text theme={null}
content-type: application/json
```

## Credits consumption

Credits are billed per operation.

| Operation | Credit cost |
| - | - |
| `POST /html` | 1 credit per request |
| `POST /screenshot` | 1 credit per request |
| `POST /spa-extract` | 1 credit per request |
| Cloud Browser (running) | 1 credit per started minute, rounded up |

Credits are deducted after request validation but before execution. Failed browser tasks do not refund credits. Cloud browser billing ends at the first confirmed stop.

## Common status codes

The API uses standard HTTP response codes.

| Code | Meaning | Cause |
| - | - | - |
| `200` | Success | Request completed normally. |
| `400` | Bad request | Invalid JSON, malformed URL, oversized body (> 1 MiB), or unsupported parameter. |
| `401` | Unauthenticated | Missing or invalid `x-api-key`. |
| `402` | Payment required | Insufficient credits (`INSUFFICIENT_CREDITS`) or a paid plan is required (`PAID_PLAN_REQUIRED`). |
| `403` | Forbidden | The referenced session was created by a different API key. |
| `404` | Not found | The requested profile or session does not exist. |
| `409` | Conflict | Cloud browser concurrency limit reached (`CLOUD_BROWSER_CONCURRENCY_LIMIT`). |
| `422` | Unprocessable | Content selector not found or readability extraction failed. |
| `429` | Too many requests | Rate limit or per-key session limit exceeded. Back off with exponential backoff. |
| `502` | Bad gateway | Proxy unreachable or the target server returned an error. |
| `503` | Service unavailable | Runtime or managed proxy temporarily unavailable (`CDP_WORKER_UNAVAILABLE`, `DYNAMIC_PROXY_NOT_CONFIGURED`). |
| `504` | Gateway timeout | Navigation or proxy connection timed out. |
| `500` | Internal error | Unclassified server failure. Retry with exponential backoff. |

## Endpoint groups

> [card] [Browser Tasks](/api-reference/html)
>
> Render HTML, capture screenshots, and extract SPA data with residential proxies and randomized fingerprints.

> [card] [Remote CDP](/api-reference/cdp-sessions)
>
> Create and manage remote Chrome DevTools Protocol sessions for low-level browser control.

> [card] [Cloud Browsers](/api-reference/cloud-browsers-list)
>
> Save, launch, start, and stop persistent cloud browser profiles with live interactive viewers.

> [card] [Schemas](/api-reference/schemas)
>
> View the shared request and response object definitions used across the API.


# API Schema Reference — Shared Types for All Endpoints
Source: https://docs.adscrawl.net/api-reference/schemas

Reusable schema definitions for browserSettings, proxy, fingerprint, cookies, waitFor, field, and actions across all AdsCrawl endpoints.

These schemas appear across multiple AdsCrawl endpoints. Instead of repeating field definitions in every endpoint reference, this page documents each schema once. Understanding `browserSettings`, `fingerprint`, `proxy`, `cookies`, `waitFor`, `field`, and `actions` before you start building lets you configure browser tasks, CDP sessions, and cloud browsers consistently and correctly.

***

## `browserSettings`

Browser configuration shared across CDP sessions and cloud browsers. Omitted regions use a random trusted proxy; explicitly pass `GLOBAL`, a region code, or a custom `proxy` object when you need deterministic routing.

**viewport** `object`

Browser window size.

**viewport fields**

**width** `number` *required*

Viewport width in pixels.

**height** `number` *required*

Viewport height in pixels.

**locale** `string`

Browser locale, such as `"en-US"`.

**timezoneId** `string`

IANA timezone identifier, such as `"Asia/Shanghai"`.

**geolocation** `object`

Optional geolocation coordinates to inject into the browser.

**geolocation fields**

**latitude** `number` *required*

Latitude in decimal degrees.

**longitude** `number` *required*

Longitude in decimal degrees.

**proxy** `object`

Custom proxy configuration. Cannot be combined with `countryCode`. See the [`proxy` schema](#proxy) below.

**countryCode** `string`

Managed proxy region. Use `"GLOBAL"` to select a popular region automatically, or a two-letter country code (e.g. `"FR"`) to prefer a trusted proxy in that region with dynamic fallback. Cannot be combined with `proxy`.

**userAgent** `string`

Overrides the default User-Agent string.

**userAgentMode** `"custom" | "random"`

Set to `"random"` to let the server select a User-Agent from its library. Requests without a User-Agent default to `"random"`.

**userAgentOs** `"windows" | "macos"`

Operating system used by random User-Agent mode. Defaults to `"windows"`.

**fingerprint** `object`

Browser fingerprint settings. When omitted in CDP contexts, `canvas` and `webGlImage` default to `"real"` while other signals use a coherent randomized profile. See the [`fingerprint` schema](#fingerprint) below.

**cookies** `cookies[]`

Cookies injected into the browser context before the session starts. See the [`cookies[]` schema](#cookies) below.

### Example

```json theme={null}
{
  "viewport": { "width": 1440, "height": 900 },
  "locale": "en-US",
  "timezoneId": "America/New_York",
  "countryCode": "US",
  "userAgentMode": "random",
  "userAgentOs": "macos",
  "fingerprint": {
    "canvas": "random",
    "webRtc": "forward"
  }
}
```

***

## `fingerprint`

Controls browser fingerprint signals to reduce detectability. Every field is optional. Browser-task omissions default to `"random"`. CDP `browserSettings` omissions set `canvas` and `webGlImage` to `"real"` and generate other signals from a coherent randomized profile.

> [!WARNING]
>
> When `webGl` is `"real"`, you cannot set `webGpu` or `hardware` to `"random"`. These three signals must be consistent with each other.

| Field | Accepted Values | Notes |
| - | - | - |
| `webRtc` | `"forward"` \| `"real"` \| `"disabled"` | `"forward"` uses the proxy exit address. The legacy value `"random"` is accepted as an alias for `"forward"`. |
| `webGl` | `"random"` \| `"real"` | Spoofs or preserves WebGL vendor and renderer metadata. |
| `webGpu` | `"random"` \| `"real"` \| `"disabled"` | In `"random"` mode, follows the WebGL GPU configuration. |
| `webGlImage` | `"random"` \| `"real"` | Controls WebGL image rendering noise. |
| `canvas` | `"random"` \| `"real"` | Adds or suppresses canvas fingerprint noise. |
| `audioContext` | `"random"` \| `"real"` | Adds or suppresses audio context fingerprint noise. |
| `clientRects` | `"random"` \| `"real"` | Adds or suppresses layout measurement noise. |
| `speechVoices` | `"random"` \| `"real"` | Returns an OS-matched or real speech voice list. |
| `fonts` | `"random"` \| `"real"` | Returns an OS-matched or real installed font list. |
| `hardware` | `"random"` \| `"real"` | Generates CPU thread count and device memory as a matched pair. |
| `doNotTrack` | `"random"` \| `"enabled"` \| `"disabled"` | Controls the DNT request header preference. |

### Example

```json theme={null}
{
  "webRtc": "forward",
  "webGl": "random",
  "webGpu": "random",
  "canvas": "random",
  "audioContext": "random",
  "fonts": "random",
  "hardware": "random",
  "doNotTrack": "disabled"
}
```

***

## `proxy`

Custom proxy configuration for routing browser traffic. Provide either the `server` URL form **or** the split form (`protocol` + `host` + `port`) — never both.

> [!WARNING]
>
> Supply `username` and `password` together. Omit both for an unauthenticated proxy. Never embed credentials in the `server` URL.

**server** `string`

Full proxy URL, e.g. `"http://host:port"` or `"socks5://host:port"`. Must include an explicit port. Cannot be combined with `host`. Do not embed credentials, a path, a query string, or a fragment.

**protocol** `"http" | "socks5"`

Proxy protocol for the split form.

**host** `string`

Proxy hostname for the split form. Cannot be combined with `server`.

**port** `number | string`

Proxy port for the split form. Accepts an integer or numeric string from 1 to 65535.

**username** `string`

Proxy authentication username. Must be supplied together with `password`.

**password** `string`

Proxy authentication password. Must be supplied together with `username`.

### Examples

```json URL form theme={null}
{
  "server": "http://proxy.example.com:8080",
  "username": "<proxy-user>",
  "password": "<proxy-password>"
}
```

```json Split form (SOCKS5) theme={null}
{
  "protocol": "socks5",
  "host": "proxy.example.com",
  "port": 1080,
  "username": "<proxy-user>",
  "password": "<proxy-password>"
}
```

```json Unauthenticated theme={null}
{
  "server": "http://proxy.example.com:8080"
}
```

***

## `cookies[]`

An array of cookies injected into the browser context before navigation begins. Each entry is an object with the following fields.

**name** `string` *required*

Cookie name.

**value** `string` *required*

Cookie value.

**domain** `string` *required*

Target domain, e.g. `".example.com"`. Include the leading dot to match subdomains.

**path** `string`

Cookie path. Defaults to `"/"`.

**secure** `boolean`

When `true`, the cookie is sent only over HTTPS.

**httpOnly** `boolean`

When `true`, the cookie is inaccessible to client-side JavaScript.

**hostOnly** `boolean`

When `true`, the cookie is bound to the exact host and not subdomains.

**sameSite** `string`

SameSite attribute. Accepted values: `"Strict"`, `"Lax"`, `"None"`.

**session** `boolean`

When `true`, the cookie expires with the session and has no persistent expiry.

**expirationDate / expires / expiry** `number`

Persistent expiry as a Unix timestamp in seconds. All three field names are accepted interchangeably.

### Example

```json theme={null}
[
  {
    "name": "sid",
    "value": "<session-token>",
    "domain": ".example.com",
    "path": "/",
    "secure": true,
    "httpOnly": true,
    "sameSite": "Lax",
    "expirationDate": 1893456000
  }
]
```

***

## `waitFor`

Instructs the browser to pause after navigation or after executing `actions` until a specific element or text becomes visible on the page. You can combine `selector` and `text` in the same object.

**selector** `string`

CSS selector. The browser waits for the first matching element to become visible.

**text** `string`

Text content. The browser waits for the first element containing this text to become visible.

**timeoutMs** `number`

Maximum wait time in milliseconds. Defaults to `15000`. Never exceeds the remaining task timeout, regardless of the value you supply.

### Example

```json theme={null}
{
  "selector": "#search-results",
  "timeoutMs": 10000
}
```

***

## `field` (SPA Extract Field Definition)

Defines a single field to extract in a `POST /spa-extract` request. DOM fields read element content; network fields capture data from intercepted JSON responses.

**source** `"dom" | "network"` *required*

Data source for this field. Use `"dom"` to read from the page's DOM, or `"network"` to capture a matching JSON response from a network request.

**selector** `string`

CSS selector identifying the target element. Required for `source: "dom"` fields.

**value** `"text" | "html" | "attribute"`

DOM read mode. Defaults to `"text"`. Use `"html"` to get the element's inner HTML, or `"attribute"` to read a specific attribute.

**attribute** `string`

The attribute name to read when `value` is `"attribute"`. For example, `"href"` or `"data-id"`.

**urlIncludes** `string`

A substring used to match the response URL for `source: "network"` fields. The most recent matching response is used.

**path** `string`

JSONPath expression to extract a value from the matched network response body. For example, `"$.data.metrics[0].value"`.

**multiple** `boolean`

When `true`, the field returns an array of all matching DOM elements instead of just the first.

**parse** `"string" | "number" | "integer" | "boolean" | "json"`

Coerces the extracted raw value to the specified type before returning it.

**regex** `string`

A regular expression applied to the extracted value. When the pattern contains a capture group, group 1 is returned as the field value.

**required** `boolean`

When `true`, a missing or unmatched field causes the endpoint to return `422 SPA_REQUIRED_FIELDS_MISSING` instead of returning `null`.

### Examples

```json DOM field theme={null}
{
  "source": "dom",
  "selector": "h1.product-title",
  "value": "text",
  "required": true
}
```

```json DOM attribute field theme={null}
{
  "source": "dom",
  "selector": "meta[name='description']",
  "value": "attribute",
  "attribute": "content"
}
```

```json Network field theme={null}
{
  "source": "network",
  "urlIncludes": "/api/v2/metrics",
  "path": "$.data.metrics[0].value",
  "parse": "number",
  "required": true
}
```

```json Multiple elements theme={null}
{
  "source": "dom",
  "selector": "ul.results li",
  "value": "text",
  "multiple": true
}
```

***

## `actions[]`

An ordered array of page interactions executed before extraction begins in `POST /spa-extract`. Each element is an object with a `type` field that determines which action runs. Every interactive step is capped at 30 seconds.

> [accordion] wait — pause execution
>
> Pauses execution for a fixed number of milliseconds.
>
> ```json theme={null}
> { "type": "wait", "milliseconds": 2000 }
> ```
>
> **type** `string` *required*
>
> Must be `"wait"`.
>
> **milliseconds** `number` *required*
>
> Duration to pause. Accepted range: `0` to `30000`.

> [accordion] waitForSelector — wait for an element
>
> Pauses until the first element matching the CSS selector becomes visible.
>
> ```json theme={null}
> { "type": "waitForSelector", "selector": "#content", "timeoutMs": 8000 }
> ```
>
> **type** `string` *required*
>
> Must be `"waitForSelector"`.
>
> **selector** `string` *required*
>
> CSS selector to wait for.
>
> **timeoutMs** `number`
>
> Maximum wait in milliseconds. Defaults to the step's 30-second cap when omitted.

> [accordion] click — click an element
>
> Clicks the first element matching the CSS selector.
>
> ```json theme={null}
> { "type": "click", "selector": "button.load-more" }
> ```
>
> **type** `string` *required*
>
> Must be `"click"`.
>
> **selector** `string` *required*
>
> CSS selector of the element to click.

> [accordion] fill — fill an input
>
> Clears the target input element and types the given value into it.
>
> ```json theme={null}
> { "type": "fill", "selector": "input[name='q']", "value": "browser automation" }
> ```
>
> **type** `string` *required*
>
> Must be `"fill"`.
>
> **selector** `string` *required*
>
> CSS selector of the input element to fill.
>
> **value** `string` *required*
>
> Text to enter into the input.

> [accordion] press — press a key
>
> Sends a keyboard key press to the first element matching the CSS selector.
>
> ```json theme={null}
> { "type": "press", "selector": "input[name='q']", "key": "Enter" }
> ```
>
> **type** `string` *required*
>
> Must be `"press"`.
>
> **selector** `string` *required*
>
> CSS selector of the target element.
>
> **key** `string` *required*
>
> Key to press, such as `"Enter"`, `"Tab"`, or `"ArrowDown"`.

> [accordion] scroll — scroll the page or an element
>
> Scrolls an element into view, or scrolls the page by a given pixel offset.
>
> ```json theme={null}
> { "type": "scroll", "y": 1200 }
> ```
>
> ```json theme={null}
> { "type": "scroll", "selector": ".lazy-section" }
> ```
>
> **type** `string` *required*
>
> Must be `"scroll"`.
>
> **selector** `string`
>
> CSS selector. When provided, the matching element is scrolled into view.
>
> **x** `number`
>
> Horizontal scroll offset in pixels. Defaults to `0`.
>
> **y** `number`
>
> Vertical scroll offset in pixels. Defaults to `800` when no `selector` is given.

### Full actions example

```json theme={null}
[
  { "type": "waitForSelector", "selector": "#cookie-banner button", "timeoutMs": 5000 },
  { "type": "click", "selector": "#cookie-banner button" },
  { "type": "fill", "selector": "input[name='q']", "value": "adscrawl api" },
  { "type": "press", "selector": "input[name='q']", "key": "Enter" },
  { "type": "waitForSelector", "selector": "#results", "timeoutMs": 10000 },
  { "type": "scroll", "y": 1600 }
]
```

***

## `cloudBrowser.runtime`

The runtime status object returned by cloud browser endpoints. It describes the current lifecycle state of a running or stopped browser session.

> [!WARNING]
>
> Never construct `connectUrl` or `cdpBaseUrl` yourself, and never append API keys, cookies, or proxy credentials to these URLs. Always use the exact URLs returned by the API. If a URL is absent from the response, query the current status before proceeding.

**runtimeKind** `"neko" | "worker_cdp"`

The type of runtime backing this browser session. `"neko"` provides an interactive browser and omits `cdpBaseUrl`. `"worker_cdp"` exposes a CDP endpoint and may include `cdpBaseUrl`.

**status** `"starting" | "running" | "stopping" | "stopped"` *required*

The current lifecycle status of the session.

**Status values**

| Value | Meaning |
| - | - |
| `starting` | Session is being provisioned. Running quota is already reserved. |
| `running` | Session is active. `connectUrl` may be present. |
| `stopping` | A stop was requested but is not yet confirmed. Running quota is still reserved. |
| `stopped` | Session has ended. Running quota is released. |

**sessionId** `string`

The active session identifier. Present for `starting`, `running`, and `stopping` states.

**expiresAt** `string (RFC3339)`

The time at which the active session expires.

**connectUrl** `string`

Direct URL to open the interactive browser. Only present when `status` is `"running"` and `runtimeKind` is `"neko"`. Requires the profile owner's login session — authorization uses the session cookie, not the viewer's `usr`/`pwd` URL parameters.

**cdpBaseUrl** `string`

The CDP base URL with a temporary token. Only returned by `worker_cdp` runtimes.

> [!NOTE]
>
> Both `"starting"` and `"stopping"` states count against the user's concurrent running quota. A `"stopping"` session has not yet released its slot — poll `GET /cloud-browsers/{id}` until you see `"stopped"` before assuming capacity is available.

### Example responses

```json Running (neko) theme={null}
{
  "runtimeKind": "neko",
  "status": "running",
  "sessionId": "<session-id>",
  "expiresAt": "2026-09-07T09:00:00.000Z",
  "connectUrl": "https://api.adscrawl.net/cloud-browser-runtime/<session-id>/?usr=adscrawl&pwd=adscrawl"
}
```

```json Stopped theme={null}
{
  "runtimeKind": "neko",
  "status": "stopped"
}
```

```json Stopping theme={null}
{
  "runtimeKind": "neko",
  "status": "stopping",
  "sessionId": "<session-id>",
  "expiresAt": "2026-09-07T09:00:00.000Z"
}
```

***

## Google Trends Result Metadata

Additional top-level fields returned alongside the standard SPA result by successful `google-trends-explore` responses. These fields describe whether the result was collected live or served from the server cache.

**source** `"sunbrowser" | "cache"` *required*

Indicates whether the result was collected from a live browser session (`"sunbrowser"`) or served from the server-side cache (`"cache"`).

**cached** `boolean` *required*

`true` when this response was served from the cache.

**stale** `boolean` *required*

`true` when the cached result is stale. The response was served from cache but the data may be outdated.

**collectedAt** `string (RFC3339Nano)` *required*

The timestamp at which the result was actually collected, in RFC3339Nano format.

**attempts** `integer` *required*

The number of collection attempts made before the result was returned. Cache HIT responses always report `0`. Stale responses report the number of attempts already made when the result was reported.

### Example

```json theme={null}
{
  "source": "sunbrowser",
  "cached": false,
  "stale": false,
  "collectedAt": "2026-09-07T08:42:11.348291200Z",
  "attempts": 1
}
```


# Capture Screenshot
Source: https://docs.adscrawl.net/api-reference/screenshot

POST /screenshot
Render any URL in a real browser and return a full-page or element-scoped PNG screenshot, routed through residential proxies.

Use `POST /screenshot` to capture a pixel-perfect PNG of any public webpage. Request a full-page capture or target a specific element with a CSS selector. The capture is performed in a real browser with randomized fingerprints and residential proxy routing so you see exactly what a real visitor sees.

**url** `string` *required*

Target page URL. Must be a reachable HTTP(S) URL using port 80 or 443.

**viewport** `object`

Viewport dimensions used when rendering and capturing the page.

**viewport fields**

**width** `number`

Viewport width in pixels.

**height** `number`

Viewport height in pixels.

**fullPage** `boolean`

Whether to capture the full scrollable page. Defaults to `true`. When `selector` is provided, only the matched element is captured regardless of this value.

**selector** `string`

CSS selector for the element to capture. Only the first matching element is screenshotted. Returns `422 CONTENT_SELECTOR_NOT_FOUND` if the selector is not found on the page.

**waitUntil** `string`

Navigation event to wait for before capturing. Defaults to `"load"`.

| Value | Behaviour |
| - | - |
| `"domcontentloaded"` | Waits for DOMContentLoaded. HTML is parsed without waiting for secondary resources such as images. |
| `"load"` | Waits for `window.load` after the page and dependent resources (images, stylesheets) finish loading. |
| `"networkidle"` | Waits until there are no network connections for at least 500 ms. Long polling, analytics, or lazy-loaded resources may cause a timeout. |

**timeoutMs** `number`

Maximum time to wait for the task to complete, in milliseconds. Must be a positive integer no greater than `3,600,000`. Values outside this range fall back to the server default.

**locale** `string`

Browser locale, for example `"en-US"` or `"zh-CN"`. Affects `navigator.language` and `Accept-Language` headers.

**timezoneId** `string`

IANA timezone identifier, for example `"Asia/Shanghai"` or `"America/New_York"`.

**geolocation** `object`

Geolocation coordinates exposed to the page via the Geolocation API.

**geolocation fields**

**latitude** `number`

Latitude in decimal degrees.

**longitude** `number`

Longitude in decimal degrees.

**proxy** `object`

Custom proxy configuration. Cannot be combined with `countryCode`. Provide either `server` or the split form (`protocol` + `host` + `port`). Credentials must not be embedded in `server`.

**proxy fields**

**server** `string`

Full proxy URL such as `http://host:port` or `socks5://host:port`. Cannot be combined with `host`.

**protocol** `string`

`"http"` or `"socks5"`. Used in the split form.

**host** `string`

Proxy host. Used in the split form.

**port** `number | string`

Port from 1 to 65535.

**username** `string`

Proxy username. Must be supplied together with `password`.

**password** `string`

Proxy password. Must be supplied together with `username`.

**countryCode** `string`

Managed residential proxy region. Cannot be combined with `proxy`.

* `"GLOBAL"`: dynamic exit from 15 popular regions.
* Two-letter country code (e.g. `"US"`, `"DE"`): prefers a trusted proxy for that region with dynamic fallback.
* Omitted: a random trusted proxy is selected automatically.

**userAgentMode** `string`

`"random"` lets the server pick a User-Agent from its library. Requests without an explicit `userAgent` already default to `"random"`.

**userAgentOs** `string`

Operating system used when `userAgentMode` is `"random"`. Accepted values: `"windows"` (default) or `"macos"`.

**userAgent** `string`

Explicit User-Agent string. Overrides the random selection.

**fingerprint** `object`

Browser fingerprint settings. When omitted, every signal defaults to random while keeping OS, GPU, CPU, memory, fonts, and device signals coherent.

**fingerprint fields**

**webRtc** `string`

`"forward"` uses the proxy exit address. `"real"` or `"disabled"` also accepted.

**webGl** `string`

`"random"` or `"real"`. Controls WebGL vendor and renderer metadata.

**webGpu** `string`

`"random"`, `"real"`, or `"disabled"`. Random mode follows the WebGL GPU setting.

**webGlImage** `string`

`"random"` or `"real"`. Controls WebGL image noise.

**canvas** `string`

`"random"` or `"real"`. Controls canvas noise.

**audioContext** `string`

`"random"` or `"real"`. Controls audio fingerprint noise.

**clientRects** `string`

`"random"` or `"real"`. Controls layout measurement noise.

**speechVoices** `string`

`"random"` or `"real"`. Returns an OS-matched speech voice list.

**fonts** `string`

`"random"` or `"real"`. Returns an OS-matched font list.

**hardware** `string`

`"random"` or `"real"`. Generates a coherent CPU thread count and memory pair.

**doNotTrack** `string`

`"random"`, `"enabled"`, or `"disabled"`.

**cookies** `array`

Cookie list injected into the browser context before navigation. Each cookie object requires `name`, `value`, and `domain`.

**cookie fields**

**name** `string` *required*

Cookie name.

**value** `string` *required*

Cookie value.

**domain** `string` *required*

Target domain such as `.example.com`.

**path** `string`

Cookie path. Defaults to `/`.

**secure** `boolean`

Whether the cookie is sent only over HTTPS.

**httpOnly** `boolean`

Whether the cookie is inaccessible to client-side JavaScript.

**sameSite** `string`

SameSite attribute.

**session** `boolean`

Set `true` for a session cookie.

**expirationDate** `number`

Unix expiry timestamp in seconds. Also accepted as `expires` or `expiry`.

## Response

| Code | Meaning |
| - | - |
| `200` | Success. Returns an `image/png` binary stream of the captured page or element. |
| `400` | Invalid JSON, URL, cookies, proxy, region, or User-Agent. Body over 1 MiB. |
| `401` | Missing or invalid `x-api-key`. |
| `402` | Insufficient balance. Returns `INSUFFICIENT_CREDITS`, `balance`, and `requiredCredits`. |
| `422` | Selector not found (`CONTENT_SELECTOR_NOT_FOUND`) or invalid Worker payload. |
| `429` | Task rate limited. |
| `502` | Proxy unreachable or target HTTP failure. |
| `503` | Queue, Worker, managed proxy, or User-Agent resources unavailable. |
| `504` | Task, navigation, or proxy connection timed out. |
| `500` | Unclassified task execution failure. |

> [!NOTE]
>
> When using `selector`, the server waits for the element to appear before capturing it. If the element is not found within the navigation timeout, the response is `422 CONTENT_SELECTOR_NOT_FOUND`.

> [!NOTE]
>
> One credit is consumed after request validation but before the task is enqueued. Request bodies are limited to 1 MiB.

```bash cURL — full page theme={null}
curl -sS -X POST "https://api.adscrawl.net/screenshot" \
  -H "content-type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{
    "url": "https://example.com",
    "viewport": { "width": 1440, "height": 900 },
    "fullPage": true,
    "waitUntil": "load",
    "countryCode": "GLOBAL",
    "userAgentMode": "random",
    "userAgentOs": "windows"
  }' \
  --output page.png
```

```bash cURL — element screenshot theme={null}
curl -sS -X POST "https://api.adscrawl.net/screenshot" \
  -H "content-type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{
    "url": "https://example.com",
    "selector": "#hero",
    "waitUntil": "load",
    "countryCode": "US"
  }' \
  --output hero.png
```

```javascript JavaScript theme={null}
import fs from "fs";

const response = await fetch("https://api.adscrawl.net/screenshot", {
  method: "POST",
  headers: {
    "content-type": "application/json",
    "x-api-key": "YOUR_API_KEY",
  },
  body: JSON.stringify({
    url: "https://example.com",
    viewport: { width: 1440, height: 900 },
    fullPage: true,
    waitUntil: "load",
    countryCode: "GLOBAL",
    userAgentMode: "random",
    userAgentOs: "windows",
  }),
});

if (!response.ok) {
  const err = await response.json();
  throw new Error(`${response.status} ${err.code}: ${err.error}`);
}

const buffer = Buffer.from(await response.arrayBuffer());
fs.writeFileSync("page.png", buffer);
```

```python Python theme={null}
import httpx

response = httpx.post(
    "https://api.adscrawl.net/screenshot",
    headers={
        "content-type": "application/json",
        "x-api-key": "YOUR_API_KEY",
    },
    json={
        "url": "https://example.com",
        "viewport": {"width": 1440, "height": 900},
        "fullPage": True,
        "waitUntil": "load",
        "countryCode": "GLOBAL",
        "userAgentMode": "random",
        "userAgentOs": "windows",
    },
)

response.raise_for_status()
with open("page.png", "wb") as f:
    f.write(response.content)
```

```text 200 image/png theme={null}
HTTP/1.1 200 OK
Content-Type: image/png

<binary PNG stream>
```

```json 422 Selector not found theme={null}
{
  "error": "Content selector was not found",
  "code": "CONTENT_SELECTOR_NOT_FOUND"
}
```

```json 402 Insufficient credits theme={null}
{
  "error": "Insufficient credits",
  "code": "INSUFFICIENT_CREDITS",
  "balance": 0,
  "requiredCredits": 1
}
```

```json 400 Bad country code theme={null}
{
  "error": "countryCode is not supported",
  "code": "INVALID_COUNTRY_CODE"
}
```

```json 503 Proxy unavailable theme={null}
{
  "error": "Dynamic country/region routing is unavailable",
  "code": "DYNAMIC_PROXY_NOT_CONFIGURED"
}
```


# Similar websites API
Source: https://docs.adscrawl.net/api-reference/similar-sites

POST https://api.adscrawl.net/similar-sites
Find related websites with POST /similar-sites. Send a URL with your API key, then use website details and relevance scores to build a research shortlist.

AdsCrawl Similar websites analyzes a website's business with AI, then searches the web for related businesses. Send a target URL to receive up to eight recommendations ranked by relevance, with the target website's details and a match score for each candidate.

## Authentication

**x-api-key** `string` *required*

Your AdsCrawl API key. Keep it in your server-side application and send it in the request header.

Send the request body as JSON with `Content-Type: application/json`.

## Request body

**url** `string`

The website to use as the starting point for discovery. The examples use `https://adspower.net/`.

**timeoutMs** `number`

Request timeout in milliseconds. The console's integration example uses `45000` (45 seconds).

`45000` is an example setting, not a documented default or limit. Give your HTTP client more time than the requested capture timeout; the examples below allow 60 seconds.

## Request examples

Set `ADSCRAWL_API_KEY` in your local shell or application environment before running an example. Replace the sample URL with the website you want to research.

```bash cURL theme={null}
curl --fail-with-body --max-time 60 \
  --request POST 'https://api.adscrawl.net/similar-sites' \
  --header "x-api-key: ${ADSCRAWL_API_KEY}" \
  --header 'Content-Type: application/json' \
  --data '{"url":"https://adspower.net/","timeoutMs":45000}'
```

```python Python theme={null}
import json
import os
import urllib.request

request = urllib.request.Request(
    "https://api.adscrawl.net/similar-sites",
    data=json.dumps({
        "url": "https://adspower.net/",
        "timeoutMs": 45000,
    }).encode("utf-8"),
    headers={
        "x-api-key": os.environ["ADSCRAWL_API_KEY"],
        "Content-Type": "application/json",
    },
    method="POST",
)
with urllib.request.urlopen(request, timeout=60) as response:
    print(json.loads(response.read()))
```

```javascript Node.js theme={null}
const response = await fetch("https://api.adscrawl.net/similar-sites", {
  method: "POST",
  headers: {
    "x-api-key": process.env.ADSCRAWL_API_KEY,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    url: "https://adspower.net/",
    timeoutMs: 45000,
  }),
  signal: AbortSignal.timeout(60000),
});
if (!response.ok) {
  throw new Error(`HTTP ${response.status}: ${await response.text()}`);
}
console.log(await response.json());
```

## Response fields

The console's integration panel identifies these fields. Inspect the returned JSON before mapping it into your application's data model.

| Field | Meaning |
| - | - |
| `target.url` | Target website URL. |
| `target.title` | Target website title. |
| `target.description` | Target website description. |
| `target.logo` | Logo information associated with the target website. |
| `candidates[].url` | Recommended website URL. |
| `candidates[].title` | Recommended website title. |
| `candidates[].score` | Relevance match score, shown from 0 to 100. |

A match score helps you prioritize further research. It is not market share, a search ranking or proof that a website is a direct competitor. Recommendations are a shortlist, not a complete market inventory.

## Repeat discovery over time

Call this endpoint from your own scheduler and save each recommendation list. Review new domains, then collect selected public pages with a separate web scraping request to compare pricing, product updates or campaigns. Scheduling, snapshot comparison and alerts belong in your application; they are not fields returned by this discovery endpoint.

You can also explore a request in the [Similar websites console](https://app.adscrawl.net/dashboard/similar-sites/). Open **Integration details** for the request example and choose **Download JSON** to inspect an existing result.


# Extract SPA Data
Source: https://docs.adscrawl.net/api-reference/spa-extract

POST /spa-extract
Extract structured data from JavaScript-heavy SPAs using custom CSS or network fields, built-in templates, or inspect mode to discover extraction candidates.

Submit a page URL and extraction configuration to retrieve structured data from a JavaScript-rendered SPA. You can use custom field definitions, a built-in template, or `inspect` mode to discover what data is available on a page.

**url** `string`

Full HTTP(S) URL for the target SPA using port 80 or 443. Required for most templates. For `google-trends-explore`, prefer the `keyword` field instead. SimilarWeb requires a full page URL, not a bare domain.

**keyword** `string`

Recommended for `google-trends-explore`. Provide 1 to 5 unique, non-empty comma-separated keywords, each at most 100 Unicode characters (for example, `"adspower,playwright"`). The server builds the canonical Trends URL automatically. Explicit `null`, numeric, or empty-string values return `INVALID_KEYWORD` and never fall back to `url`.

**mode** `string`

Extraction mode. Defaults to `"extract"`.

| Value | Behaviour |
| - | - |
| `"extract"` | Runs field or template extraction and returns structured `data`. |
| `"inspect"` | Returns DOM and network candidates discovered on the page along with a `suggestedPlan` you can copy into a future extract request. |

**template** `string`

Built-in site template ID. When provided, `fields` are supplied by the template rather than your request. Available template IDs:

* `"similarweb-overview"` — website traffic, engagement, and ranking metrics.
* `"google-trends-explore"` — interest-over-time data and relative averages for up to 5 keywords.
* `"chrome-web-store-app-info"` — extension metadata from the Chrome Web Store (with CRX manifest fallback).

**parameters** `object`

Template-specific parameters declared by the selected template. Refer to the template's `input` specification from `GET /spa-extract/templates`.

**waitUntil** `string`

Navigation event to wait for before extraction. Custom extraction defaults to `"domcontentloaded"`. Site templates use the request value first, then the template default, then `"domcontentloaded"`.

| Value | Behaviour |
| - | - |
| `"domcontentloaded"` | Waits for DOMContentLoaded — recommended for most SPAs. |
| `"load"` | Waits for `window.load` including images and stylesheets. |
| `"networkidle"` | Waits until no network connections for 500 ms. May timeout on long-polling sites. |

**waitFor** `object`

Wait for a specific element or text to appear after SPA navigation and actions complete. `selector` and `text` may be combined.

**waitFor fields**

**selector** `string`

Wait for the first matching element to become visible.

**text** `string`

Wait for the first element containing this text to become visible.

**timeoutMs** `number`

Timeout in milliseconds. Defaults to 15,000 and never exceeds the remaining task timeout.

**actions** `array`

Optional page interactions executed in array order before extraction. Each step is capped at 30 seconds.

**action types**

**wait** `object`

`{ type: "wait", milliseconds: number }` — Pause for 0 to 30,000 ms.

**waitForSelector** `object`

`{ type: "waitForSelector", selector: string, timeoutMs?: number }` — Wait for an element to become visible.

**click** `object`

`{ type: "click", selector: string }` — Click the first matching element.

**fill** `object`

`{ type: "fill", selector: string, value: string }` — Clear and fill the first matching input.

**press** `object`

`{ type: "press", selector: string, key: string }` — Press a key on the first matching element.

**scroll** `object`

`{ type: "scroll", selector?: string, x?: number, y?: number }` — Scroll an element into view or scroll the page. `y` defaults to 800.

**fields** `object`

Record of named field definitions for `extract` mode. Each key becomes a property in the response `data` object. Use this when you are not using a template.

**field definition**

**source** `string` *required*

`"dom"` to read from the page HTML, or `"network"` to read from intercepted JSON responses.

**selector** `string`

CSS selector for a DOM field.

**value** `string`

DOM read mode: `"text"` (default), `"html"`, or `"attribute"`.

**attribute** `string`

Attribute name, used when `value` is `"attribute"`.

**urlIncludes** `string`

Substring to match against a response URL for a network field.

**path** `string`

JSON path within the matched network response, for example `$.data.metrics[0].value`.

**multiple** `boolean`

Whether a DOM field should return all matching elements as an array.

**parse** `string`

Coerce the value to `"string"`, `"number"`, `"integer"`, `"boolean"`, or `"json"`.

**regex** `string`

Optional regular expression. When a capture group is present, group 1 is used.

**required** `boolean`

Missing required fields return `422 SPA_REQUIRED_FIELDS_MISSING`.

**schema** `object`

Legacy alias for `fields`, used only when `fields` is absent. String values are treated as DOM CSS selectors. Does not validate the response JSON and does not return `schemaValid` or `schemaErrors` fields. Prefer `fields` for all new integrations.

**timeoutMs** `number`

Maximum time to wait for the task to complete, in milliseconds. Must be a positive integer no greater than `3,600,000`.

**viewport** `object`

Viewport dimensions used when rendering the page.

**viewport fields**

**width** `number`

Viewport width in pixels.

**height** `number`

Viewport height in pixels.

**locale** `string`

Browser locale, for example `"en-US"`. May follow trusted proxy metadata when omitted.

**timezoneId** `string`

IANA timezone identifier, for example `"Asia/Shanghai"`. May follow trusted proxy metadata when omitted.

**geolocation** `object`

Geolocation coordinates exposed to the page via the Geolocation API.

**geolocation fields**

**latitude** `number`

Latitude in decimal degrees.

**longitude** `number`

Longitude in decimal degrees.

**proxy** `object`

Custom proxy configuration. Cannot be combined with `countryCode`.

**proxy fields**

**server** `string`

Full proxy URL such as `http://host:port` or `socks5://host:port`. Cannot be combined with `host`.

**protocol** `string`

`"http"` or `"socks5"`. Used in the split form.

**host** `string`

Proxy host. Used in the split form.

**port** `number | string`

Port from 1 to 65535.

**username** `string`

Proxy username. Must be supplied together with `password`.

**password** `string`

Proxy password. Must be supplied together with `username`.

**countryCode** `string`

Managed residential proxy region. Cannot be combined with `proxy`.

* `"GLOBAL"` — dynamic exit from 15 popular regions.
* Two-letter country code — prefers a trusted proxy with dynamic fallback.
* Omitted — a random trusted proxy is selected automatically.

**userAgentMode** `string`

`"random"` lets the server pick a User-Agent from its library. Requests without an explicit `userAgent` default to `"random"`.

**userAgentOs** `string`

Operating system used when `userAgentMode` is `"random"`. Accepted values: `"windows"` (default) or `"macos"`.

**userAgent** `string`

Explicit User-Agent string. Overrides the random selection.

**fingerprint** `object`

Browser fingerprint settings. When omitted, every signal defaults to random while keeping OS, GPU, CPU, memory, fonts, and device signals coherent.

**fingerprint fields**

**webRtc** `string`

`"forward"`, `"real"`, or `"disabled"`.

**webGl** `string`

`"random"` or `"real"`. Controls WebGL vendor and renderer metadata.

**webGpu** `string`

`"random"`, `"real"`, or `"disabled"`.

**webGlImage** `string`

`"random"` or `"real"`. Controls WebGL image noise.

**canvas** `string`

`"random"` or `"real"`. Controls canvas noise.

**audioContext** `string`

`"random"` or `"real"`. Controls audio fingerprint noise.

**clientRects** `string`

`"random"` or `"real"`. Controls layout measurement noise.

**speechVoices** `string`

`"random"` or `"real"`. Returns an OS-matched speech voice list.

**fonts** `string`

`"random"` or `"real"`. Returns an OS-matched font list.

**hardware** `string`

`"random"` or `"real"`. Generates a coherent CPU thread count and memory pair.

**doNotTrack** `string`

`"random"`, `"enabled"`, or `"disabled"`.

**cookies** `array`

Cookie list injected into the browser context before navigation.

> [!WARNING]
>
> `google-trends-explore` ignores omitted, `null`, or empty-array values. Any non-empty or malformed cookies on a Trends request return `400 INVALID_TRENDS_COOKIES`.

**cookie fields**

**name** `string` *required*

Cookie name.

**value** `string` *required*

Cookie value.

**domain** `string` *required*

Target domain such as `.example.com`.

**path** `string`

Cookie path. Defaults to `/`.

**secure** `boolean`

Whether the cookie is sent only over HTTPS.

**httpOnly** `boolean`

Whether the cookie is inaccessible to client-side JavaScript.

**sameSite** `string`

SameSite attribute.

**session** `boolean`

Set `true` for a session cookie.

**expirationDate** `number`

Unix expiry timestamp in seconds. Also accepted as `expires` or `expiry`.

## Response

**mode** `string`

`"extract"` or `"inspect"`, mirroring the request mode.

**page** `object`

Page metadata.

**page fields**

**url** `string`

Final page URL.

**title** `string`

Page title.

**data** `object`

Extracted structured data. Keys match your `fields` definition or the template's output fields.

**missingFields** `array`

List of field keys that were not found.

**source** `string`

`"sunbrowser"` — result came from a live browser collection. `"cache"` — result was served from the server cache. Present on Trends responses.

**cached** `boolean`

Whether this response was served from cache.

**stale** `boolean`

Whether the cached result is stale.

**collectedAt** `string`

RFC3339Nano timestamp of when the result was actually collected.

**attempts** `number`

Collection attempt count. Cache HIT responses use `0`; stale responses use the attempt count at the time the Worker reported them.

**candidates** `object`

Present in `inspect` mode. DOM and network extraction candidates discovered on the page.

**suggestedPlan** `object`

Present in `inspect` mode. A plan with `fields` and `schema` you can copy into a future `extract` request.

| Code | Meaning |
| - | - |
| 200 | Inspection results or extracted structured data. |
| 400 | Invalid URL, keyword, proxy, region, User-Agent, or Trends cookies. |
| 401 | Missing or invalid `x-api-key`. |
| 402 | Insufficient balance (`INSUFFICIENT_CREDITS`). |
| 404 | Chrome Web Store listing unavailable and CRX fallback could not recover it. |
| 422 | Required fields missing, invalid template or field config, or Worker rejected the task. |
| 429 | Task rate limited, or Google Trends upstream 429. |
| 502 | Proxy or target failure, invalid Trends data, or all candidate identities exhausted. |
| 503 | Queue, Worker, or managed resources unavailable; or Trends identity plan expired. |
| 504 | Task, navigation, or proxy timeout; or Trends response not observed. |
| 500 | Unclassified task execution failure. |

> [!NOTE]
>
> One credit is consumed after request validation but before the task is enqueued. Request bodies are limited to 1 MiB.

## More examples

```bash Custom fields theme={null}
curl -sS -X POST "https://api.adscrawl.net/spa-extract" \
  -H "content-type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{
    "url": "https://example.com/dashboard",
    "mode": "extract",
    "waitUntil": "domcontentloaded",
    "waitFor": { "selector": "h1", "timeoutMs": 15000 },
    "fields": {
      "title":    { "source": "dom", "selector": "h1",           "parse": "string" },
      "subtitle": { "source": "dom", "selector": "p.subtitle",   "parse": "string" },
      "count":    { "source": "dom", "selector": ".metric-value", "parse": "number" }
    },
    "countryCode": "GLOBAL",
    "userAgentMode": "random"
  }'
```

```bash SimilarWeb template theme={null}
curl -sS -X POST "https://api.adscrawl.net/spa-extract" \
  -H "content-type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{
    "template": "similarweb-overview",
    "url": "https://www.similarweb.com/website/dolphin-anty.com/#overview",
    "countryCode": "GLOBAL"
  }'
```

```bash Google Trends theme={null}
curl -sS -X POST "https://api.adscrawl.net/spa-extract" \
  -H "content-type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{
    "template": "google-trends-explore",
    "keyword": "adspower,playwright"
  }'
```

```bash Inspect mode theme={null}
curl -sS -X POST "https://api.adscrawl.net/spa-extract" \
  -H "content-type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{
    "url": "https://example.com/dashboard",
    "mode": "inspect"
  }'
```

```bash cURL theme={null}
curl -sS -X POST "https://api.adscrawl.net/spa-extract" \
  -H "content-type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{
    "url": "https://example.com/dashboard",
    "mode": "extract",
    "waitUntil": "domcontentloaded",
    "waitFor": { "selector": "h1", "timeoutMs": 15000 },
    "fields": {
      "title":    { "source": "dom", "selector": "h1",           "parse": "string" },
      "subtitle": { "source": "dom", "selector": "p.subtitle",   "parse": "string" },
      "count":    { "source": "dom", "selector": ".metric-value", "parse": "number" }
    },
    "countryCode": "GLOBAL",
    "userAgentMode": "random"
  }'
```

```javascript JavaScript theme={null}
const response = await fetch("https://api.adscrawl.net/spa-extract", {
  method: "POST",
  headers: {
    "content-type": "application/json",
    "x-api-key": "YOUR_API_KEY",
  },
  body: JSON.stringify({
    url: "https://example.com/dashboard",
    mode: "extract",
    waitUntil: "domcontentloaded",
    waitFor: { selector: "h1", timeoutMs: 15000 },
    fields: {
      title:    { source: "dom", selector: "h1",            parse: "string" },
      subtitle: { source: "dom", selector: "p.subtitle",    parse: "string" },
      count:    { source: "dom", selector: ".metric-value", parse: "number" },
    },
    countryCode: "GLOBAL",
    userAgentMode: "random",
  }),
});

const result = await response.json();
console.log(result.data);
```

```python Python theme={null}
import httpx

response = httpx.post(
    "https://api.adscrawl.net/spa-extract",
    headers={
        "content-type": "application/json",
        "x-api-key": "YOUR_API_KEY",
    },
    json={
        "url": "https://example.com/dashboard",
        "mode": "extract",
        "waitUntil": "domcontentloaded",
        "waitFor": {"selector": "h1", "timeoutMs": 15000},
        "fields": {
            "title":    {"source": "dom", "selector": "h1",             "parse": "string"},
            "subtitle": {"source": "dom", "selector": "p.subtitle",     "parse": "string"},
            "count":    {"source": "dom", "selector": ".metric-value",  "parse": "number"},
        },
        "countryCode": "GLOBAL",
        "userAgentMode": "random",
    },
)

response.raise_for_status()
print(response.json()["data"])
```

```json 200 Extract mode theme={null}
{
  "mode": "extract",
  "page": {
    "url": "https://example.com/dashboard",
    "title": "Example Dashboard"
  },
  "data": {
    "title": "Example Dashboard"
  },
  "missingFields": []
}
```

```json 200 Inspect mode theme={null}
{
  "mode": "inspect",
  "page": {
    "url": "https://example.com/dashboard",
    "title": "Example Dashboard"
  },
  "candidates": {
    "dom": { "metrics": [], "tables": [] },
    "network": []
  },
  "suggestedPlan": {
    "fields": {},
    "schema": {
      "type": "object",
      "properties": {},
      "additionalProperties": false
    }
  }
}
```

```json 200 Google Trends theme={null}
{
  "mode": "extract",
  "page": {
    "url": "https://trends.google.com/trends/explore?q=adspower%2Cplaywright",
    "title": "Google Trends"
  },
  "data": {
    "averages": [
      { "query": "adspower",    "extractedValue": 42 },
      { "query": "playwright",  "extractedValue": 71 }
    ],
    "interestOverTime": []
  },
  "missingFields": [],
  "source": "sunbrowser",
  "cached": false,
  "stale": false,
  "collectedAt": "2026-08-03T02:04:05.123456789Z",
  "attempts": 2
}
```

```json 402 Payment required theme={null}
{
  "code": "INSUFFICIENT_CREDITS"
}
```

```json 422 Required fields missing theme={null}
{
  "code": "SPA_REQUIRED_FIELDS_MISSING"
}
```


# List Templates
Source: https://docs.adscrawl.net/api-reference/spa-extract-templates

GET /spa-extract/templates
Retrieve the list of available built-in extraction templates with their input types, example URLs, expected output fields, and metadata.

Retrieve the list of available built-in extraction templates. Each template includes its required input type, example URL, expected output fields, and metadata.

## Response

**templates** `array`

List of available extraction templates.

**template fields**

**id** `string`

Template ID used in POST /spa-extract requests.

**name** `string`

Human-readable template name.

**description** `string`

What the template extracts.

**urlPattern** `string`

Regex pattern matching supported URLs.

**exampleUrl** `string`

Example page URL for this template.

**version** `number`

Template version.

**updatedAt** `string`

Last update timestamp in ISO 8601 format.

**outputFields** `array`

List of field names the template returns.

**waitUntil** `string`

Default navigation wait event.

**input** `object`

Input specification.

**input fields**

**type** `string`

Input type, for example "domain".

**example** `string`

Example input value.

**urlTemplate** `string`

URL template with  placeholder.

**catalog** `object`

Catalog metadata.

**catalog fields**

**category** `string`

Template category.

**featured** `boolean`

Whether the template is featured.

**keywords** `array`

Search keywords.

| Code | Meaning |
| - | - |
| 200 | Active templates with their inputs, defaults, and output fields. |
| 401 | Missing or invalid `x-api-key`. |
| 500 | Failed to load the template catalog. |

> [!TIP]
>
> Pass the template `id` (for example, `"similarweb-overview"`) as the `template` field in your `POST /spa-extract` request body.

```bash cURL theme={null}
curl -sS "https://api.adscrawl.net/spa-extract/templates" \
  -H "x-api-key: YOUR_API_KEY"
```

```javascript JavaScript theme={null}
const response = await fetch("https://api.adscrawl.net/spa-extract/templates", {
  headers: { "x-api-key": "YOUR_API_KEY" },
});

const result = await response.json();
console.log(result.templates);
```

```python Python theme={null}
import httpx

response = httpx.get(
    "https://api.adscrawl.net/spa-extract/templates",
    headers={"x-api-key": "YOUR_API_KEY"},
)

response.raise_for_status()
print(response.json()["templates"])
```

```json 200 OK theme={null}
{
  "templates": [
    {
      "id": "similarweb-overview",
      "name": "SimilarWeb Website Overview",
      "description": "Get public website traffic, engagement, and ranking metrics from SimilarWeb.",
      "urlPattern": "^https://www\\.similarweb\\.com/website/[^/?#]+/?(?:[?#].*)?$",
      "exampleUrl": "https://www.similarweb.com/website/dolphin-anty.com/#overview",
      "version": 5,
      "updatedAt": "2026-07-18T14:07:33.000Z",
      "outputFields": [
        "avgVisitDuration",
        "avgVisitDurationSeconds",
        "bounceRate",
        "categoryRank",
        "countryRank",
        "globalRank",
        "pagesPerVisit",
        "period",
        "targetDomain",
        "totalVisits",
        "totalVisitsDisplay"
      ],
      "waitUntil": "domcontentloaded",
      "input": {
        "type": "domain",
        "example": "dolphin-anty.com",
        "urlTemplate": "https://www.similarweb.com/website/{value}/#overview"
      },
      "catalog": {
        "category": "analytics",
        "featured": true,
        "keywords": ["traffic", "rank", "website analytics"]
      }
    }
  ]
}
```

```json 401 Unauthorized theme={null}
{
  "error": "Unauthorized"
}
```

```json 500 Server error theme={null}
{
  "error": "Failed to load template catalog"
}
```


# API Keys, Session JWTs, and CDP Tokens in AdsCrawl
Source: https://docs.adscrawl.net/authentication

Learn how to use API keys, session JWTs, and CDP data tokens to authenticate every AdsCrawl request securely from your backend server.

Every request to the AdsCrawl API must carry a credential. For nearly all workflows, that credential is your API key — a long-lived secret you send in the `x-api-key` request header. Two narrower token types exist for specific scenarios: a session JWT for cloud browser lifecycle operations initiated from a dashboard session, and a data token embedded in CDP session URLs for WebSocket access. Understanding which credential goes where prevents authentication errors and keeps your key secure.

## Getting your API key

Open the [AdsCrawl dashboard](https://app.adscrawl.net/dashboard/) after signing in, and copy your API key from the keys section. Store it in an environment variable or a secrets manager — never hard-code it in your application source.

```bash theme={null}
export ADSCRAWL_API_KEY="your-api-key"
```

## Sending your API key

Pass your API key in the `x-api-key` header on every request. The header name is lowercase; the value is the full key string.

```bash cURL theme={null}
curl --fail-with-body -sS -X POST "https://api.adscrawl.net/html" \
  -H "content-type: application/json" \
  -H "x-api-key: $ADSCRAWL_API_KEY" \
  -d '{"url": "https://example.com", "contentMode": "markdown"}'
```

```javascript JavaScript (fetch) theme={null}
const response = await fetch("https://api.adscrawl.net/html", {
  method: "POST",
  headers: {
    "content-type": "application/json",
    "x-api-key": process.env.ADSCRAWL_API_KEY,
  },
  body: JSON.stringify({
    url: "https://example.com",
    contentMode: "markdown",
  }),
});
```

```python Python theme={null}
import os
import requests

response = requests.post(
    "https://api.adscrawl.net/html",
    headers={
        "content-type": "application/json",
        "x-api-key": os.environ["ADSCRAWL_API_KEY"],
    },
    json={"url": "https://example.com", "contentMode": "markdown"},
)
```

> [!WARNING]
>
> Never expose your API key in client-side JavaScript, browser extensions, mobile apps, public repositories, or request URLs. Your key authorizes charges against your account. If a key is compromised, rotate it immediately from the dashboard.

## Session JWT (cloud browser lifecycle)

Cloud browser list, create, launch, start, and stop endpoints also accept a session JWT in the `Authorization: Bearer <SESSION_JWT>` header. This is a login session token — it represents a signed-in dashboard user, not an API key.

Session authentication is narrower than API key authentication in one important way: **it requires `apiKeyId`** on start requests. You must supply the UUID of an active, unexpired API key that belongs to the same account. API key authentication selects the current key automatically.

```bash theme={null}
curl --fail-with-body -sS -X POST \
  "https://api.adscrawl.net/cloud-browsers/<CLOUD_BROWSER_ID>/start" \
  -H "Authorization: Bearer <SESSION_JWT>" \
  -H "content-type: application/json" \
  -d '{"apiKeyId": "<API_KEY_ID>", "countryCode": "GLOBAL"}'
```

> [!NOTE]
>
> Session cookies are also accepted in place of the Bearer token for dashboard-initiated requests. Use API key authentication for all server-side and automated workflows.

## Data tokens (CDP WebSocket access)

When you create a CDP session, the `201` response includes a `cdpBaseUrl` that already contains an embedded data token:

```json theme={null}
{
  "sessionId": "6c3f7d14-7fe4-4c8e-9f1b-0b6d6f2fa2ef",
  "expiresAt": "2026-04-21T10:30:00.000Z",
  "cdpBaseUrl": "https://api.adscrawl.net/cdp/sessions/6c3f7d14-...?token=<data-token>"
}
```

Pass `cdpBaseUrl` directly to Playwright's `connectOverCDP`. The data token in this URL authorizes both the CDP discovery endpoint (`GET /cdp/sessions/:id/json/version`) and the CDP WebSocket (`WSS /cdp/sessions/:id/devtools/browser/:browserId`).

> [!WARNING]
>
> Do **not** replace the data token in `cdpBaseUrl` with your API key. The data token is a separate, session-scoped credential. Substituting your API key will result in a `401` error.

For live browser control, use `POST /cdp/live-token` (authenticated with your API key) to obtain a short-lived `controlToken`. This single-use token expires after 30 seconds and authorizes one WebSocket connection to `WSS /cdp/live/:sessionId`.

## Error reference

| HTTP Status | Meaning | What to do |
| - | - | - |
| `401` | Missing or invalid `x-api-key` | Verify the header name is `x-api-key` (lowercase) and the value is your full, untruncated key. |
| `402` | Insufficient credits | Check your credit balance in the dashboard and upgrade your plan or wait for the next renewal. |
| `403` | Session does not belong to the current key | The `sessionId` you referenced was created by a different API key. Use the key that created the session. |

## API key limits by plan

The number of API keys you can create depends on your plan. Each key is independent and can be scoped to different servers or services.

| Plan | Price | API Keys |
| - | - | - |
| Free | \$0 | 1 |
| Hobby | \$9 / month | 3 |
| Starter | \$49 / month | 10 |
| Pro | \$199 / month | 30 |

> [!NOTE]
>
> You can view and manage your API keys, credit balance, and plan from the [dashboard](https://app.adscrawl.net/dashboard/). To create a new account, visit [app.adscrawl.net/register](https://app.adscrawl.net/register/).


# Browser Tasks: HTML, Screenshots, and Data Extraction
Source: https://docs.adscrawl.net/concepts/browser-tasks

Synchronous, credit-metered requests that spin up a full browser and return rendered HTML, screenshots, or structured data — no session management needed.

Browser tasks are the simplest way to get rendered output from any URL. Each request spins up a fresh Chromium instance, navigates to your target page through a residential proxy, and returns a result synchronously — no session to create, no cleanup required. One credit is consumed after request validation but before the browser launches, so a malformed request body never costs you credits. Request bodies are limited to 1 MiB and `timeoutMs` can be set up to 3,600,000 ms (one hour) to handle slow or complex pages.

## Task types

AdsCrawl offers three browser task endpoints, each optimized for a different output:

### HTML / Markdown

`POST /html` renders the target page and returns one of three content modes:

* **`html`** (default) — the full rendered page HTML, or just the matched element when `selector` is set
* **`markdown`** — Readability-extracted article content converted to Markdown
* **`json`** — a structured Readability article object with `title`, `byline`, `excerpt`, `textContent`, `length`, and more

Use `waitUntil: "domcontentloaded"` for most HTML extraction tasks — it is faster than waiting for images and stylesheets to load. Switch to `"networkidle"` only when the content you need is fetched by XHR after page load.

### Screenshots

`POST /screenshot` captures a PNG image of the rendered page and streams it back directly as `image/png`. Key options:

* **`fullPage: true`** (default) — captures the entire scrollable page
* **`selector`** — captures only the first element matching the CSS selector; returns `422 CONTENT_SELECTOR_NOT_FOUND` if the selector does not match
* **`viewport`** — set width and height to control the rendering viewport before capture

### SPA Extraction

`POST /spa-extract` extracts structured fields from dynamic single-page applications. You can define your own fields, run page actions (click, fill, scroll) before extraction, and use `waitFor` to wait for an element or text to appear. Two modes are available:

* **`inspect`** — returns page candidates and a suggested field plan without consuming extra credits
* **`extract`** — returns the data defined in your `fields` map or a named `template`

AdsCrawl maintains a catalog of ready-made site templates (e.g., `similarweb-overview`, `google-trends-explore`, `chrome-web-store-app-info`). Retrieve the full list with `GET /spa-extract/templates`.

## How credits work

One credit is deducted per browser task request. The charge happens **after validation** (so bad requests are free) but **before the browser launches** (so navigation failures still cost one credit). Check your balance in the [dashboard](https://app.adscrawl.net/dashboard/) before running large batches.

## Proxy behavior

Every browser task runs through a proxy. You control which proxy is used with two optional fields:

| Setting | Behavior |
| - | - |
| Neither field set | Random trusted residential proxy assigned automatically |
| `countryCode: "US"` | Managed proxy preferring a trusted exit in that region, with dynamic fallback |
| `countryCode: "GLOBAL"` | Dynamic exit rotating across 15 popular regions |
| `proxy` object | Your own HTTP or SOCKS5 proxy server |

`countryCode` and `proxy` are mutually exclusive — pass one or the other, never both.

## Request limits

| Limit | Value |
| - | - |
| Request body | 1 MiB maximum |
| `timeoutMs` | 1 – 3,600,000 ms |
| Supported ports | 80 and 443 only |

> [!NOTE]
>
> Browser tasks are **stateless** — each request creates a fresh browser context with no memory of previous visits. If you need to maintain cookies, local storage, or a logged-in session across multiple pages, use [Remote CDP](/concepts/remote-cdp) or [Cloud Browsers](/concepts/cloud-browsers) instead.

## API reference

> [card] [POST /html](/api-reference/html)
>
> Fetch rendered HTML, Markdown, or structured article JSON from any URL.

> [card] [POST /screenshot](/api-reference/screenshot)
>
> Capture a full-page or element-level PNG screenshot.

> [card] [POST /spa-extract](/api-reference/spa-extract)
>
> Extract structured fields from dynamic SPAs with optional templates and actions.


# Cloud Browsers: Persistent, Reusable Browser Profiles
Source: https://docs.adscrawl.net/concepts/cloud-browsers

Save browser profiles with cookies, fingerprints, and proxy settings that persist across sessions — launch, stop, and reuse them as many times as you need.

Cloud browsers are saved browser profiles that retain their configuration — viewport, locale, fingerprint settings, cookies, and proxy preferences — between sessions. Unlike temporary Remote CDP sessions, a cloud browser profile persists after you stop the running instance. You can launch it again later, and it picks up exactly where it left off. This makes cloud browsers the right choice for workflows that require a consistent browser identity across multiple visits: account management, multi-session scraping, or any task where a persistent, recognizable browser persona matters.

## Lifecycle states

A cloud browser moves through four states during its lifetime:

```text theme={null}
starting → running → stopping → stopped
```

* **`starting`** and **`stopping`** still reserve your running allowance — they are not free slots.
* **`running`** is the only state where `connectUrl` is present in the API response.
* **`stopped`** releases running allowance while the saved profile remains.

Poll `GET /cloud-browsers/:id` to read the actual current state. Display `runtime.status` exactly as the API returns it; never infer or cache state locally.

## Saved profiles vs. running sessions

Saving a profile is free in terms of running quota — a stopped profile occupies a **saved profile slot** but not a **running slot**. Only browsers in `starting`, `running`, or `stopping` state consume your concurrent running allowance.

Your plan's allowances are returned on every `GET /cloud-browsers` call:

| Field | Meaning |
| - | - |
| `limit` | Maximum number of saved profiles your plan allows |
| `runningLimit` | Maximum number of simultaneously running browsers |
| `runningCount` | Current count of starting + running + stopping browsers |

> [!NOTE]
>
> The Free plan allows saving one profile but **cannot launch** a browser. A paid plan is required to start a cloud browser runtime.

## Opening the interactive viewer

When a cloud browser is `running`, the `runtime.connectUrl` field contains a URL you can open directly in your own browser to see and interact with the remote session in real time. This URL points to the live browser viewer and requires your profile owner's **session cookie** — an API key alone does not grant viewer access.

> [!WARNING]
>
> **Closing the viewer tab does not stop billing.** The browser continues running — and consuming credits — until you explicitly call `POST /cloud-browsers/:id/stop`. Always stop the browser through the API or dashboard when you are done.

## Two ways to create and start

### Option 1: Create then Start

Use this approach when you want to save a profile for later or when you need fine-grained control over when the browser starts.

**1. Create the profile**

`POST /cloud-browsers` saves the profile configuration and returns an `id`. The browser is **not** started yet and consumes no running quota.

```bash theme={null}
curl --fail-with-body -sS -X POST "https://api.adscrawl.net/cloud-browsers" \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "remark": "work profile",
    "browserSettings": {
      "viewport": { "width": 1440, "height": 900 }
    }
  }'
```

**2. Start the browser**

`POST /cloud-browsers/:id/start` launches the saved profile. Pass your proxy on **every** start request — saved proxy settings do not replace the required top-level `proxy` field for API key callers.

```bash theme={null}
curl --fail-with-body -sS -X POST \
  "https://api.adscrawl.net/cloud-browsers/BROWSER_ID/start" \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "proxy": {
      "server": "http://proxy.example.com:8080",
      "username": "proxy-user",
      "password": "proxy-password"
    }
  }'
```

The endpoint waits for startup confirmation and returns `200` only once the browser reaches `running`.

### Option 2: Launch in one step

`POST /cloud-browsers/launch` creates the profile **and** starts the browser in a single request, waiting for `running` before returning `201`. Use this when you want to open a browser immediately without a separate create step.

```bash theme={null}
curl --fail-with-body -sS --max-time 200 \
  -X POST "https://api.adscrawl.net/cloud-browsers/launch" \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "proxy": {
      "server": "http://proxy.example.com:8080",
      "username": "proxy-user",
      "password": "proxy-password"
    },
    "tabs": ["https://example.com"]
  }'
```

> [!WARNING]
>
> Each call to `POST /cloud-browsers/launch` creates a **new profile** — there is no idempotency key. If the request fails or the response is lost, inspect `GET /cloud-browsers` before retrying. Never automatically repeat a failed launch call without checking whether a profile was created.

## Proxy requirements

Proxy rules differ depending on how you authenticate:

> [tab] API key (X-API-Key)
>
> Every start request — including `/launch` and `/start` — **must** include a valid top-level `proxy` object. Saved `browserSettings.proxy`, a previous run's proxy, and `countryCode` cannot substitute for it.
>
> ```json theme={null}
> {
>   "proxy": {
>     "server": "http://proxy.example.com:8080",
>     "username": "proxy-user",
>     "password": "proxy-password"
>   }
> }
> ```

> [tab] Session / JWT Bearer
>
> Dashboard and JWT Bearer callers may use either a custom `proxy` object **or** `countryCode` (a two-letter region code or `"GLOBAL"`). A session-authenticated start also requires `apiKeyId` for an active API key owned by the same user.
>
> ```json theme={null}
> {
>   "apiKeyId": "YOUR_API_KEY_ID",
>   "countryCode": "GLOBAL"
> }
> ```

> [!WARNING]
>
> A proxy failure must **never** fall back to a direct connection. If your proxy is unreachable, fix it or wait for service recovery — do not remove the proxy field to work around the error.

## Credit billing

Cloud browsers are billed at **1 credit per started minute**, rounded up, from the moment the browser successfully reaches `running` until stop is confirmed. A `202 stopping` response means shutdown is still in progress — credits continue until `runtime.status` is `stopped`. Poll `GET /cloud-browsers/:id` with a bounded retry loop to confirm the stop.

Calling stop multiple times does not charge twice; repeated stops on an already-stopped profile return `200` with `runtime.status: "stopped"` at no cost.

## API reference

> [card] [List Cloud Browsers](/api-reference/cloud-browsers-list)
>
> List saved profiles and read your saved and running quotas.

> [card] [Create Profile](/api-reference/cloud-browsers-create)
>
> Save a new browser profile without starting a runtime.

> [card] [Launch](/api-reference/cloud-browsers-launch)
>
> Create and start a browser in one request, waiting for running.

> [card] [Start](/api-reference/cloud-browsers-start)
>
> Start a previously saved profile and wait for running confirmation.

> [card] [Stop](/api-reference/cloud-browsers-stop)
>
> Stop a running browser and release its running quota slot.


# Proxy Routing: Built-In, Regional, and Custom Proxies
Source: https://docs.adscrawl.net/concepts/proxies

Every AdsCrawl browser runs through a proxy. Learn how to use built-in residential proxies, route by country, or supply your own proxy server.

Every browser that AdsCrawl runs — whether a browser task, a Remote CDP session, or a cloud browser — is routed through a proxy. Residential proxies are built into the platform, so you get sensible defaults out of the box without any configuration. When you need traffic to appear from a specific country, or when your workflow requires a dedicated proxy service, AdsCrawl gives you precise control through two optional fields: `countryCode` and `proxy`.

## The three proxy modes

### Omitted — automatic trusted proxy

If you send a browser task or create a CDP session without specifying `countryCode` or `proxy`, AdsCrawl automatically assigns a random trusted residential proxy. This is the simplest option and works well when geographic location does not matter.

### `countryCode` — managed regional proxy

Set `countryCode` to a two-letter ISO region code to prefer a trusted proxy exit in that region. If no trusted proxy is available in the requested region, AdsCrawl falls back to a dynamic proxy automatically.

Set `countryCode` to `"GLOBAL"` to use a dynamic exit that rotates across 15 popular regions — useful when you want geographic diversity without pinning to one country.

```json theme={null}
{
  "url": "https://example.com/article",
  "contentMode": "markdown",
  "countryCode": "DE"
}
```

Supported country codes include:

| Code | Region |
| - | - |
| `US` | United States |
| `BR` | Brazil |
| `DE` | Germany |
| `SG` | Singapore |
| `JP` | Japan |
| `CA` | Canada |
| `AU` | Australia |
| `GLOBAL` | Dynamic — 15 popular regions |

> [!TIP]
>
> When using `countryCode`, also set `locale` and `timezoneId` to match the target region. For example, pair `"DE"` with `locale: "de-DE"` and `timezoneId: "Europe/Berlin"` so the browser's fingerprint matches the proxy exit location.

> [!NOTE]
>
> `countryCode` is **not available** for API key callers starting cloud browsers. API key cloud browser starts require an explicit `proxy` object on every request. Session/JWT Bearer callers can use `countryCode` for cloud browser starts.

### `proxy` — bring your own proxy

Provide a `proxy` object to route the browser through your own HTTP or SOCKS5 proxy server. Use this when you have a dedicated proxy service, need sticky sessions, or require a specific IP.

**HTTP proxy with credentials:**

```json theme={null}
{
  "proxy": {
    "server": "http://proxy.example.com:8080",
    "username": "proxy-user",
    "password": "proxy-password"
  }
}
```

**SOCKS5 proxy using split fields:**

```json theme={null}
{
  "proxy": {
    "protocol": "socks5",
    "host": "proxy.example.com",
    "port": 1080,
    "username": "proxy-user",
    "password": "proxy-password"
  }
}
```

**Unauthenticated proxy:**

```json theme={null}
{
  "proxy": {
    "server": "http://proxy.example.com:8080"
  }
}
```

### Custom proxy format rules

* Use `server` for a full URL (`http://host:port` or `socks5://host:port`), **or** use `protocol` + `host` + `port` — never both in the same request
* Ports must be integers or integer strings between 1 and 65535
* Do **not** embed credentials in the URL — pass `username` and `password` as separate fields
* `username` and `password` must be supplied together; omit both for an unauthenticated proxy
* No path, query string, or fragment is allowed in `server`

## Combining `countryCode` and `proxy`

`countryCode` and `proxy` are mutually exclusive. Passing both in the same request returns `400 COUNTRY_PROXY_CONFLICT`. Choose one:

```json Use countryCode theme={null}
{
  "url": "https://example.com",
  "countryCode": "JP"
}
```

```json Use proxy theme={null}
{
  "url": "https://example.com",
  "proxy": {
    "server": "http://proxy.example.com:8080",
    "username": "user",
    "password": "pass"
  }
}
```

## Example: countryCode in a POST /html request

```bash cURL theme={null}
curl -sS -X POST "https://api.adscrawl.net/html" \
  -H "content-type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{
    "url": "https://example.com/article",
    "contentMode": "markdown",
    "countryCode": "US",
    "locale": "en-US",
    "timezoneId": "America/New_York",
    "userAgentMode": "random",
    "userAgentOs": "windows"
  }'
```

```json Request body theme={null}
{
  "url": "https://example.com/article",
  "contentMode": "markdown",
  "countryCode": "US",
  "locale": "en-US",
  "timezoneId": "America/New_York",
  "userAgentMode": "random",
  "userAgentOs": "windows"
}
```

## Proxy failure behavior

> [!WARNING]
>
> **Never remove the `proxy` field to work around a proxy error.** Proxy failures must not fall back to a direct connection — AdsCrawl enforces this at the infrastructure level to protect your real IP and maintain proxy hygiene. If your proxy is unreachable, fix the configuration or wait for service recovery before retrying.

When a managed proxy allocation fails, the start or task request is rejected immediately. When a custom proxy is unreachable, the request fails with a `502` or `503` error. Inspect the error code, correct the proxy configuration, and retry.


# Remote CDP: Connect Any CDP Client to a Cloud Browser
Source: https://docs.adscrawl.net/concepts/remote-cdp

Spin up a dedicated cloud Chromium session and connect to it with Playwright, Puppeteer, or raw CDP — fully managed, no infrastructure required.

Remote CDP gives you a dedicated, managed Chromium session that you connect to using any CDP-compatible automation library. Create a session through the API, receive a `cdpBaseUrl`, and pass it directly to `playwright.connectOverCDP()` or Puppeteer's `connect()`. From there, every Playwright or Puppeteer API you already know works exactly as it does locally — navigate, click, fill forms, intercept network requests, and extract data — while AdsCrawl handles the browser infrastructure, proxies, and fingerprinting.

## How it works

**1. Create a session**

Send `POST /cdp/sessions` with your desired `idleTimeoutMs`, `maxSessionMs`, and `browserSettings`. The API responds with a `sessionId` and a `cdpBaseUrl` that already contains your data token.

```bash theme={null}
curl -sS -X POST "https://api.adscrawl.net/cdp/sessions" \
  -H "content-type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{
    "idleTimeoutMs": 600000,
    "maxSessionMs": 3600000,
    "browserSettings": {
      "viewport": { "width": 1440, "height": 900 },
      "countryCode": "GLOBAL",
      "userAgentMode": "random",
      "userAgentOs": "windows"
    }
  }'
```

The response looks like:

```json theme={null}
{
  "sessionId": "6c3f7d14-7fe4-4c8e-9f1b-0b6d6f2fa2ef",
  "expiresAt": "2026-04-21T10:30:00.000Z",
  "cdpBaseUrl": "https://api.adscrawl.net/cdp/sessions/6c3f7d14-7fe4-4c8e-9f1b-0b6d6f2fa2ef?token=<data-token>"
}
```

**2. Connect with Playwright or Puppeteer**

Pass `cdpBaseUrl` directly to your automation library. No additional configuration is needed.

```typescript Playwright (TypeScript) theme={null}
import { chromium } from 'playwright';

const browser = await chromium.connectOverCDP(
  'https://api.adscrawl.net/cdp/sessions/SESSION_ID?token=DATA_TOKEN'
);
const page = await browser.newPage();
await page.goto('https://example.com');
const title = await page.title();
console.log(title);
await browser.close();
```

```javascript Puppeteer (JavaScript) theme={null}
import puppeteer from 'puppeteer-core';

const browser = await puppeteer.connect({
  browserWSEndpoint:
    'wss://api.adscrawl.net/cdp/sessions/SESSION_ID/devtools/browser/BROWSER_ID?token=DATA_TOKEN',
});
const page = await browser.newPage();
await page.goto('https://example.com');
console.log(await page.title());
await browser.close();
```

```javascript Raw CDP (JavaScript) theme={null}
const discovery = await fetch(
  'https://api.adscrawl.net/cdp/sessions/SESSION_ID/json/version?token=DATA_TOKEN'
).then((res) => res.json());

const socket = new WebSocket(discovery.webSocketDebuggerUrl);
```

**3. Automate the page**

Navigate, interact, and extract data using the full Playwright or Puppeteer API surface — the remote browser behaves identically to a local one.

```typescript theme={null}
const page = await browser.newPage();
await page.goto('https://example.com/login');
await page.fill('#email', 'user@example.com');
await page.fill('#password', 'secret');
await page.click('button[type="submit"]');
await page.waitForNavigation();
const accountName = await page.textContent('.account-name');
```

**4. Delete the session when done**

Call `DELETE /cdp/sessions/:sessionId` to terminate the browser and release resources immediately, rather than waiting for the idle or max-session timeout to expire.

```bash theme={null}
curl -sS -X DELETE "https://api.adscrawl.net/cdp/sessions/SESSION_ID" \
  -H "x-api-key: YOUR_API_KEY"
```

## Security model

Remote CDP uses two separate tokens to protect your session:

* **Data token** — embedded in `cdpBaseUrl` and used for all CDP WebSocket connections and discovery requests. This token is long-lived (valid for the session lifetime) and scoped to read/control the browser over CDP.
* **Control token** (`controlToken`) — a single-use token issued by `POST /cdp/live-token` for live interactive viewer access. It expires after **30 seconds** and can only be consumed once. Request a fresh control token each time you need to open the live viewer.

> [!WARNING]
>
> Never put your data token or API key in client-side code, browser URLs accessible to end users, or public logs. Treat the data token with the same care as your API key.

## Session lifecycle

Configure the session lifetime at creation time with two fields:

| Field | Description |
| - | - |
| `idleTimeoutMs` | Terminates the session if no CDP activity is detected for this duration. Values above the server cap are clamped. |
| `maxSessionMs` | Hard upper limit on total session duration regardless of activity. Values above the server cap are clamped. |

Query active sessions with `GET /cdp/sessions` at any time, or delete a specific session with `DELETE /cdp/sessions/:sessionId`.

## Remote CDP vs. Cloud Browsers

Remote CDP sessions are **temporary** — they hold no saved profile, and all browser state (cookies, local storage, history) is discarded when the session ends. Use Remote CDP when you need a stateful scripted session without long-term profile persistence.

If you need to save a browser profile with cookies, fingerprint settings, and proxy configuration that persists across sessions, see [Cloud Browsers](/concepts/cloud-browsers) instead.

## API reference

> [card] [CDP Sessions](/api-reference/cdp-sessions)
>
> Create, list, and delete dedicated CDP sessions.

> [card] [CDP Live Token](/api-reference/cdp-live-token)
>
> Issue a single-use control token for live interactive viewer access.


# AdsCrawl Credits, Rate Limits, and Concurrency Caps
Source: https://docs.adscrawl.net/credits-and-limits

Understand how AdsCrawl credits are consumed, how rate limits and concurrency caps apply, and which error codes signal billing or capacity issues.

Every AdsCrawl plan comes with a credit allowance and a set of limits that govern how many requests you can make, how many browser sessions you can run simultaneously, and how many Cloud Browser profiles you can keep saved. Understanding these boundaries helps you design reliable automations that handle errors gracefully rather than hitting unexpected walls in production.

## Credit Consumption

Credits are the billing unit for every operation on AdsCrawl. The table below shows how each API type consumes them.

| Operation | Credit cost | When consumed |
| - | - | - |
| `POST /html` | 1 credit per request | After request validation, before execution |
| `POST /screenshot` | 1 credit per request | After request validation, before execution |
| `POST /spa-extract` | 1 credit per request | After request validation, before execution |
| Cloud Browser (running) | 1 credit per started minute | From successful start until confirmed stop |

> [!WARNING]
>
> Credits are **non-refundable** once consumed. If a browser task request fails during execution — after the credit has already been deducted — the credit is not returned. Validate your inputs before sending requests to avoid unnecessary credit loss.

> [!NOTE]
>
> Cloud Browser billing is per **started minute**, rounded up. A browser that runs for 90 seconds consumes 2 credits. Repeated stop notifications do not trigger duplicate charges — billing ends at the first confirmed stop.

## Rate Limits

AdsCrawl enforces per-key rate limits to protect service stability. When you exceed a rate limit, the API returns **HTTP 429**. Back off and retry after a delay.

> [!TIP]
>
> Implement **exponential backoff** when you receive a 429 response. Start with a 1-second delay and double it on each retry, up to a reasonable maximum (for example, 30 seconds), before giving up.

### CDP Session Rate Limits

Remote CDP sessions have a per-API-key concurrency limit. Exceeding it returns:

```json theme={null}
{
  "error": "CDP sessions per API key limit reached"
}
```

with HTTP status **429** and the code `SESSIONS_PER_API_KEY_LIMIT_REACHED`. Delete an existing session before creating a new one, or distribute load across multiple API keys if your plan allows.

## Concurrency Limits

### Cloud Browser Concurrency

Each user account has a `runningLimit` — the maximum number of Cloud Browser sessions (in `starting`, `running`, or `stopping` state) that can be active at the same time. The default is **1**.

You can inspect your current usage at any time via `GET /cloud-browsers`:

```json theme={null}
{
  "limit": 10,
  "runningLimit": 1,
  "runningCount": 0
}
```

* **`limit`** — your plan's saved Cloud Browser profile allowance
* **`runningLimit`** — maximum simultaneous running sessions for your account
* **`runningCount`** — sessions currently in `starting`, `running`, or `stopping` state, across all profiles, pages, and API keys

> [!NOTE]
>
> `runningCount` includes sessions across **all** your API keys and dashboard activity. A session in `stopping` state still occupies a running slot until it reaches `stopped`.

When you try to start a Cloud Browser and the running allowance is full, the API returns **409 CLOUD\_BROWSER\_CONCURRENCY\_LIMIT**:

```json theme={null}
{
  "error": "Cloud browser running limit reached",
  "code": "CLOUD_BROWSER_CONCURRENCY_LIMIT"
}
```

> [!WARNING]
>
> **Lowering your `runningLimit` does not stop existing sessions.** It only prevents new starts until running sessions free up capacity. Stop a running browser before attempting a new start when you are at your limit.

## Error Codes Reference

The following error codes indicate billing or capacity problems. Use the HTTP status code and the stable `code` field for error handling in your application — do not rely on the human-readable `error` message, which may change.

| HTTP | Code | Meaning | Action |
| - | - | - | - |
| 402 | `INSUFFICIENT_CREDITS` | Your credit balance is zero or too low to complete the request | Top up your balance in [Billing & Usage](https://app.adscrawl.net/dashboard/) |
| 402 | `PAID_PLAN_REQUIRED` | This feature requires an active paid plan | Upgrade your plan from the [dashboard](https://app.adscrawl.net/dashboard/) |
| 409 | `CLOUD_BROWSER_CONCURRENCY_LIMIT` | Your concurrent running limit is full or set to zero | Stop a running browser before starting another |
| 429 | *(varies)* | Rate limit or per-key session limit exceeded | Wait and retry with exponential backoff |

## Request Size and Parameter Limits

The following hard limits apply to all API requests. Requests that exceed these bounds are rejected with HTTP 400 before any credit is consumed.

| Limit | Value |
| - | - |
| Request body size | 1 MiB maximum |
| `timeoutMs` parameter | 3,600,000 ms (1 hour) maximum |
| Cloud Browser tabs on launch (`tabs` array) | 8 URLs maximum |
| Cookie list (`cookies` array) | Up to 10,000 entries, within the 1 MiB request body cap |


# Create and Manage Persistent Cloud Browser Sessions
Source: https://docs.adscrawl.net/guides/cloud-browser-sessions

Create persistent cloud browser profiles with saved cookies and fingerprints, launch them on demand, and view them live in the interactive browser.

Cloud browsers give you a persistent profile — stored cookies, a consistent fingerprint, and saved proxy preferences — that you can launch repeatedly without reconfiguring from scratch each time. Every start resumes exactly where you left off: the browser opens with your injected cookies already set and your fingerprint already applied.

## Launch a cloud browser

> [tab] Quick Launch
>
> `POST /cloud-browsers/launch` creates a profile and waits until the browser is running before returning. Use this when you want a single request that gives you a ready-to-use `connectUrl`.
>
> ```bash theme={null}
> curl --fail-with-body -sS --max-time 200 \
>   -X POST 'https://api.adscrawl.net/cloud-browsers/launch' \
>   -H 'x-api-key: YOUR_API_KEY' \
>   -H 'content-type: application/json' \
>   -d '{
>     "proxy": {
>       "server": "http://proxy.example.com:8080",
>       "username": "user",
>       "password": "pass"
>     },
>     "tabs": ["https://example.com"],
>     "cookies": [{
>       "name": "session",
>       "value": "abc123",
>       "domain": "example.com",
>       "path": "/"
>     }]
>   }'
> ```
>
> A successful `201` response includes a `connectUrl` you can open directly:
>
> ```json theme={null}
> {
>   "ok": true,
>   "id": "<browser-id>",
>   "source": "launch",
>   "deleteOnStop": false,
>   "runtime": {
>     "runtimeKind": "neko",
>     "status": "running",
>     "sessionId": "<session-id>",
>     "expiresAt": "2026-09-07T09:00:00.000Z",
>     "connectUrl": "https://api.adscrawl.net/cloud-browser-runtime/<session-id>/?usr=adscrawl&pwd=adscrawl"
>   }
> }
> ```
>
> > [!WARNING]
> >
> > Each call to `/cloud-browsers/launch` creates a **new** profile. If the request fails or the response is lost, inspect `GET /cloud-browsers` before retrying — never automatically repeat the launch POST.

> [tab] Create then Start
>
> Use two separate requests when you want more control — for example, to save a profile now and start it later, or to verify the profile was created before spending a running slot.
>
> **Step 1 — Create and save the profile:**
>
> ```bash theme={null}
> curl --fail-with-body -sS \
>   -X POST 'https://api.adscrawl.net/cloud-browsers' \
>   -H 'x-api-key: YOUR_API_KEY' \
>   -H 'content-type: application/json' \
>   -d '{
>     "remark": "work profile",
>     "browserSettings": {
>       "viewport": { "width": 1440, "height": 900 }
>     }
>   }'
> ```
>
> Response: `{ "ok": true, "id": "<browser-id>" }`
>
> **Step 2 — Start the profile (always send the proxy again):**
>
> ```bash theme={null}
> curl --fail-with-body -sS --max-time 65 \
>   -X POST 'https://api.adscrawl.net/cloud-browsers/<browser-id>/start' \
>   -H 'x-api-key: YOUR_API_KEY' \
>   -H 'content-type: application/json' \
>   -d '{
>     "proxy": {
>       "server": "http://proxy.example.com:8080",
>       "username": "user",
>       "password": "pass"
>     }
>   }'
> ```
>
> > [!NOTE]
> >
> > Every API key start request must include an explicit `proxy` object, even if the profile already has a saved proxy. The saved `browserSettings.proxy` does not substitute for it.

## Open the interactive browser viewer

Once `runtime.status` is `"running"`, open the `connectUrl` in any browser where you're signed in to your AdsCrawl account. The viewer lets you control the remote browser interactively in real time.

> [!WARNING]
>
> **Closing the viewer tab does not stop the cloud browser.** The session continues running and billing continues at one credit per started minute. Always call `POST /cloud-browsers/:id/stop` explicitly when you're done.

## Stop a cloud browser

Send a `POST` to stop the running session:

```bash theme={null}
curl --fail-with-body -sS --max-time 65 \
  -X POST 'https://api.adscrawl.net/cloud-browsers/<browser-id>/stop' \
  -H 'x-api-key: YOUR_API_KEY'
```

The response status tells you whether stop is confirmed:

| HTTP status | `runtime.status` | Meaning |
| - | - | - |
| `200` | `stopped` | Session is fully stopped; running slot released |
| `202` | `stopping` | Shutdown in progress; quota still reserved |
| `409` | — | Startup still in progress; wait and retry |

When you receive `202`, poll the profile detail endpoint until `stopped` is confirmed:

```bash theme={null}
# Poll until stopped (example — add a bound in production)
until [ "$(curl -sS 'https://api.adscrawl.net/cloud-browsers/<browser-id>' \
  -H 'x-api-key: YOUR_API_KEY' | jq -r '.runtime.status')" = "stopped" ]; do
  echo "Still stopping..."
  sleep 2
done
echo "Stopped."
```

## Check concurrency limits

Retrieve your current quota usage alongside the profile list:

```bash theme={null}
curl -sS 'https://api.adscrawl.net/cloud-browsers?page=1&pageSize=10' \
  -H 'x-api-key: YOUR_API_KEY' | jq '{limit, runningLimit, runningCount}'
```

| Field | Meaning |
| - | - |
| `limit` | Maximum number of saved profiles on your plan |
| `runningLimit` | Maximum simultaneous running sessions for your account |
| `runningCount` | Sessions currently in `starting`, `running`, or `stopping` state |

> [!NOTE]
>
> If you exceed `runningLimit`, the start or launch request returns `409 CLOUD_BROWSER_CONCURRENCY_LIMIT`. Stop an active session to free a slot before retrying. This is distinct from `409 Cloud Browser limit reached`, which means your saved-profile allowance is full.

## Profile retention and reuse

> [!TIP]
>
> `deleteOnStop` defaults to `false`, so your profile — including saved cookies and tab snapshots — is retained after every stop. You can restart the same profile as many times as you like, or query it while stopped, without losing your accumulated session state.

Profiles are only removed when you call `DELETE /cloud-browsers/:id` explicitly. A stopped profile still counts toward your saved-profile `limit` but does not consume any running slot.

## Runtime status reference

> [accordion] Runtime status lifecycle
>
> **starting** `status`
>
> The browser is initialising. The running slot is reserved. Do not attempt another start on this profile; wait for `running` or handle the error.
>
> **running** `status`
>
> The browser is live. `connectUrl` is present in the runtime object. Billing is active.
>
> **stopping** `status`
>
> A stop has been requested but is not yet confirmed. The running slot is still reserved. Poll until `stopped`.
>
> **stopped** `status`
>
> The session has ended. The running slot is released. The saved profile and its cookies are retained.


# Extract Structured Data from JavaScript-Heavy Pages
Source: https://docs.adscrawl.net/guides/extract-data

Use POST /spa-extract to pull typed fields from SPAs via DOM selectors or network interception, or apply a pre-built template for popular sites.

Use `POST /spa-extract` to pull specific, typed fields out of single-page applications and other JavaScript-heavy sites. You can target elements directly with CSS selectors, intercept live network responses to capture API payloads, or apply a pre-built template for popular sites like SimilarWeb and Google Trends — without writing or maintaining a scraper.

## Two extraction modes

The `mode` field controls what the endpoint does:

| Mode | What it returns |
| - | - |
| `inspect` | A suggested extraction plan: candidate DOM selectors and intercepted network calls found on the page |
| `extract` | The actual field values, extracted according to your `fields` definition or a named `template` |

## Discover fields with inspect mode

Run `inspect` first when you're working with an unfamiliar page. The response shows you which DOM elements and network responses are available, and generates a `suggestedPlan` you can paste directly into an `extract` request.

```json theme={null}
{
  "url": "https://example.com/dashboard",
  "mode": "inspect"
}
```

The response looks like this:

```json theme={null}
{
  "mode": "inspect",
  "page": {
    "url": "https://example.com/dashboard",
    "title": "Example Dashboard"
  },
  "candidates": {
    "dom": { "metrics": [], "tables": [] },
    "network": []
  },
  "suggestedPlan": {
    "fields": {},
    "schema": {
      "type": "object",
      "properties": {},
      "additionalProperties": false
    }
  }
}
```

## Extract custom fields

Define a `fields` object in your request body, where each key is the output field name and its value describes where and how to read it.

### DOM fields

A DOM field reads from a rendered element. Set `source: "dom"`, point to an element with `selector`, and choose a `parse` type to coerce the value.

```json theme={null}
{
  "url": "https://example.com/dashboard",
  "mode": "extract",
  "waitFor": { "selector": "h1", "timeoutMs": 15000 },
  "fields": {
    "title": { "source": "dom", "selector": "h1", "parse": "string" },
    "price": { "source": "dom", "selector": ".price", "parse": "number" }
  }
}
```

### Network fields

A network field intercepts a JSON response from the page's own API calls. Set `source: "network"`, use `urlIncludes` to match the request URL, and supply a `path` using JSONPath syntax to reach the value inside the response.

```json theme={null}
{
  "url": "https://example.com/dashboard",
  "mode": "extract",
  "fields": {
    "visitorCount": {
      "source": "network",
      "urlIncludes": "/api/metrics",
      "path": "$.data.visitors"
    }
  }
}
```

### Field options reference

> [accordion] field definition options
>
> **source** `"dom" | "network"` *required*
>
> Data source for the field.
>
> **selector** `string`
>
> CSS selector (DOM fields only).
>
> **value** `"text" | "html" | "attribute"`
>
> DOM read mode. Defaults to `"text"`. Use `"attribute"` together with the `attribute` field.
>
> **urlIncludes** `string`
>
> Substring to match a network response URL (network fields only).
>
> **path** `string`
>
> JSONPath expression into the matched response, e.g. `$.data.metrics[0].value`.
>
> **parse** `"string" | "number" | "integer" | "boolean" | "json"`
>
> Coerce the extracted value to this type.
>
> **multiple** `boolean`
>
> Return all matching elements as an array (DOM fields only).
>
> **regex** `string`
>
> Apply a regular expression. When a capture group is present, group 1 is returned.
>
> **required** `boolean`
>
> Missing required fields cause a `422 SPA_REQUIRED_FIELDS_MISSING` response.

## Use pre-built templates

AdsCrawl ships three maintained templates for sites that are difficult to scrape reliably. Pass `template` instead of `fields`.

> [card] similarweb-overview
>
> Website traffic, engagement, and ranking metrics from SimilarWeb.

> [card] google-trends-explore
>
> Interest-over-time and average values for up to five search terms.

> [card] chrome-web-store-app-info
>
> Extension metadata including ratings, user counts, and categories.

List templates and see their full output schemas at any time:

```bash theme={null}
curl -sS "https://api.adscrawl.net/spa-extract/templates" \
  -H "x-api-key: $ADSCRAWL_API_KEY"
```

### SimilarWeb example

Pass the full SimilarWeb URL and a `countryCode` for proxy routing. The template handles authentication, scroll timing, and field extraction automatically.

```json theme={null}
{
  "template": "similarweb-overview",
  "url": "https://www.similarweb.com/website/example.com/#overview",
  "countryCode": "GLOBAL"
}
```

### Google Trends example

Supply a comma-separated list of keywords via the `keyword` field (recommended). The API builds the canonical Trends URL for you. You can compare up to five unique terms.

```json theme={null}
{
  "template": "google-trends-explore",
  "keyword": "playwright,puppeteer"
}
```

> [!NOTE]
>
> Do not pass `cookies` to a Google Trends request. Any non-empty cookies array returns `400 INVALID_TRENDS_COOKIES`.

## Run actions before extraction

Use the `actions` array to interact with the page — click buttons, fill forms, scroll, or wait — before fields are extracted. Actions execute in order and each step is capped at 30 seconds.

```json theme={null}
{
  "url": "https://example.com/dashboard",
  "mode": "extract",
  "actions": [
    { "type": "click", "selector": "#load-more" },
    { "type": "wait", "milliseconds": 1500 },
    { "type": "scroll", "y": 800 }
  ],
  "fields": {
    "title": { "source": "dom", "selector": "h1", "parse": "string" },
    "price": { "source": "dom", "selector": ".price", "parse": "number" }
  }
}
```

Available action types:

| Type | Description |
| - | - |
| `wait` | Pause for 0–30,000 ms |
| `waitForSelector` | Wait for an element to become visible |
| `click` | Click the first matching element |
| `fill` | Clear and type into an input |
| `press` | Press a keyboard key on an element |
| `scroll` | Scroll an element into view or the page by `x`/`y` |

## Full extract response

A successful `extract` response always contains `mode`, `page`, `data`, and `missingFields`:

```json theme={null}
{
  "mode": "extract",
  "page": {
    "url": "https://example.com/dashboard",
    "title": "Example Dashboard"
  },
  "data": {
    "title": "Example Dashboard",
    "price": 49.99
  },
  "missingFields": []
}
```

`missingFields` lists any field names that were defined but could not be resolved. Fields marked `required: true` that are missing will instead return a `422` error.


# Fetch Rendered HTML, Markdown, and JSON from Any Page
Source: https://docs.adscrawl.net/guides/fetch-html

Use POST /html to retrieve fully rendered page content as raw HTML, clean Markdown, or structured JSON — no browser management required.

Use `POST /html` to fetch the fully rendered DOM of any page without spinning up or maintaining a browser yourself. AdsCrawl runs a real Chromium instance behind residential proxies, waits for the page to finish loading, and returns the content in whichever format your workflow needs.

## Choose a content mode

The `contentMode` field controls what you get back:

| Mode | Response type | Best for |
| - | - | - |
| `html` *(default)* | `text/html` | Full rendered DOM, scraping, link extraction |
| `markdown` | `text/markdown` | LLM pipelines, readable article text |
| `json` | `application/json` | Structured article data with `title`, `byline`, `excerpt`, and `content` fields |

`markdown` and `json` modes both use the [Mozilla Readability](https://github.com/mozilla/readability) algorithm. If the page has no recognisable article content, the API returns `422 READABILITY_CONTENT_NOT_FOUND`.

## Send your first request

**1. Choose your contentMode**

Decide whether you need the full page DOM (`html`), clean readable text (`markdown`), or a structured article object (`json`). For most LLM and summarisation tasks, `markdown` is the right starting point.

**2. Set waitUntil**

Use `domcontentloaded` for speed — it returns as soon as HTML is parsed without waiting for images and stylesheets to load. Switch to `load` when you need styles to be applied, or `networkidle` when the page makes follow-up XHR requests to fill in content.

**3. Optionally scope with selector**

Set `selector` to a CSS selector to wait for that element and return only its HTML (in `html` mode) or run Readability against it (in `markdown`/`json` mode). A selector that doesn't match returns `422`.

**4. Send the request**

Include your API key in the `x-api-key` header and POST a JSON body to `https://api.adscrawl.net/html`.

```bash cURL theme={null}
curl -sS -X POST "https://api.adscrawl.net/html" \
  -H "content-type: application/json" \
  -H "x-api-key: $ADSCRAWL_API_KEY" \
  -d '{
    "url": "https://example.com/article",
    "contentMode": "markdown",
    "waitUntil": "domcontentloaded"
  }'
```

```javascript JavaScript theme={null}
const response = await fetch('https://api.adscrawl.net/html', {
  method: 'POST',
  headers: {
    'content-type': 'application/json',
    'x-api-key': process.env.ADSCRAWL_API_KEY,
  },
  body: JSON.stringify({
    url: 'https://example.com/article',
    contentMode: 'markdown',
    waitUntil: 'domcontentloaded',
  }),
});
const text = await response.text();
```

```python Python theme={null}
import os, requests

resp = requests.post(
    'https://api.adscrawl.net/html',
    headers={'x-api-key': os.environ['ADSCRAWL_API_KEY']},
    json={
        'url': 'https://example.com/article',
        'contentMode': 'markdown',
        'waitUntil': 'domcontentloaded',
    }
)
print(resp.text)
```

## JSON response (contentMode=json)

When you use `contentMode: "json"`, the response body is a structured Readability article object:

```json theme={null}
{
  "title": "Example Article",
  "byline": "Author Name",
  "excerpt": "A concise article summary.",
  "siteName": "Example",
  "lang": "en",
  "dir": null,
  "content": "<div><p>Readable body...</p></div>",
  "textContent": "Readable body...",
  "length": 2487,
  "publishedTime": null
}
```

| Field | Description |
| - | - |
| `title` | Article headline extracted by Readability |
| `byline` | Author or publication credit |
| `excerpt` | Short summary, often from a meta description |
| `siteName` | Publisher name |
| `lang` | Detected language code |
| `dir` | Text direction (`"ltr"`, `"rtl"`, or `null`) |
| `content` | Cleaned HTML body |
| `textContent` | Plain-text body, whitespace-normalised |
| `length` | Character count of `textContent` |
| `publishedTime` | Publication timestamp if found, otherwise `null` |

## Tips and common options

> [!TIP]
>
> Set `countryCode` to a two-letter region code (e.g. `"US"`, `"GB"`) or `"GLOBAL"` to route your request through a residential proxy in that region. This lets you bypass geo-restrictions and retrieve the localised version of a page.

> [!TIP]
>
> Pass a `cookies` array to fetch pages that require an authenticated session. Each cookie needs at minimum `name`, `value`, and `domain`.

> [!NOTE]
>
> If you request `contentMode: "markdown"` or `contentMode: "json"` on a page that contains no recognisable article body — such as a login screen or a data-heavy dashboard — the API returns `422 READABILITY_CONTENT_NOT_FOUND`. Switch to `contentMode: "html"` to retrieve the raw DOM for those pages.

## Full request body reference

> [accordion] All request fields
>
> **url** `string` *required*
>
> Target page URL. Only ports 80 and 443 are supported.
>
> **contentMode** `"html" | "markdown" | "json"`
>
> Output format. Defaults to `"html"`. `"markdown"` and `"json"` apply Readability to extract the article body.
>
> **selector** `string`
>
> CSS selector. Waits for the first matching element; `html` mode returns only that element's HTML. `markdown`/`json` modes run Readability against it. Missing selectors return `422`.
>
> **waitUntil** `"domcontentloaded" | "load" | "networkidle"`
>
> Navigation wait condition. Defaults to `"load"`. Use `"domcontentloaded"` for faster HTML extraction.
>
> **countryCode** `string`
>
> Managed proxy region. `"GLOBAL"` picks a dynamic exit node from 15 popular regions. A two-letter code (e.g. `"US"`) prefers a trusted proxy with dynamic fallback. Cannot be combined with a custom `proxy`.
>
> **cookies** `cookies[]`
>
> Cookie list injected into the browser context before navigation.
>
> **locale** `string`
>
> Browser locale, such as `"en-US"` or `"zh-CN"`.
>
> **timezoneId** `string`
>
> IANA timezone ID, such as `"Asia/Shanghai"`.
>
> **viewport** `object`
>
> Viewport size, e.g. `{ "width": 1280, "height": 720 }`.
>
> **timeoutMs** `number`
>
> Navigation timeout in milliseconds. Must be positive and no greater than 3,600,000.


# Automate Browsers with Playwright via Remote CDP Sessions
Source: https://docs.adscrawl.net/guides/playwright-automation

Connect Playwright or Puppeteer to a remote AdsCrawl cloud browser over CDP to run full multi-step automation through residential proxies.

Connect Playwright or Puppeteer to a dedicated cloud browser over the Chrome DevTools Protocol (CDP) for multi-step automation that goes beyond single-page fetching. AdsCrawl provisions a real Chromium instance behind a residential proxy, returns a `cdpBaseUrl` you connect to directly, and charges only for the time the session is active.

## Full automation flow

**1. Create a CDP session**

Send a `POST /cdp/sessions` request with your desired browser settings. The API returns a `sessionId` and a `cdpBaseUrl` that includes an embedded data token.

```bash theme={null}
curl -sS -X POST "https://api.adscrawl.net/cdp/sessions" \
  -H "content-type: application/json" \
  -H "x-api-key: $ADSCRAWL_API_KEY" \
  -d '{
    "idleTimeoutMs": 300000,
    "maxSessionMs": 1800000,
    "browserSettings": {
      "countryCode": "US",
      "userAgentMode": "random",
      "viewport": { "width": 1440, "height": 900 }
    }
  }'
```

Response:

```json theme={null}
{
  "sessionId": "6c3f7d14-7fe4-4c8e-9f1b-0b6d6f2fa2ef",
  "expiresAt": "2026-04-21T10:30:00.000Z",
  "cdpBaseUrl": "https://api.adscrawl.net/cdp/sessions/6c3f7d14-7fe4-4c8e-9f1b-0b6d6f2fa2ef?token=<data-token>"
}
```

**2. Connect Playwright using cdpBaseUrl**

Pass `cdpBaseUrl` directly to `chromium.connectOverCDP()`. Playwright handles the WebSocket handshake; you don't need to construct any URLs manually.

**3. Automate: navigate, click, fill, wait**

Use the full Playwright API — `page.goto()`, `page.fill()`, `page.click()`, `page.waitForSelector()`, and everything else. The browser runs in AdsCrawl's infrastructure, routed through the proxy you configured.

**4. Collect your results**

Read DOM content, take screenshots, capture network responses, or export cookies and storage — whatever your automation needs.

**5. Close the browser and delete the session**

Call `browser.close()` to release Playwright's connection, then `DELETE /cdp/sessions/:sessionId` to terminate the cloud browser and stop billing.

## TypeScript example

```typescript TypeScript (Playwright) theme={null}
import { chromium } from 'playwright';

async function main() {
  // 1. Create a CDP session
  const sessionRes = await fetch('https://api.adscrawl.net/cdp/sessions', {
    method: 'POST',
    headers: {
      'content-type': 'application/json',
      'x-api-key': process.env.ADSCRAWL_API_KEY!,
    },
    body: JSON.stringify({
      idleTimeoutMs: 300000,
      maxSessionMs: 1800000,
      browserSettings: {
        countryCode: 'US',
        userAgentMode: 'random',
        viewport: { width: 1440, height: 900 },
      },
    }),
  });
  const session = await sessionRes.json();

  // 2. Connect Playwright
  const browser = await chromium.connectOverCDP(session.cdpBaseUrl);
  const page = await browser.newPage();

  // 3. Automate
  await page.goto('https://example.com/login');
  await page.fill('#email', 'user@example.com');
  await page.fill('#password', 'secret');
  await page.click('button[type=submit]');
  await page.waitForSelector('.dashboard');

  // 4. Collect results
  const title = await page.title();
  console.log('Dashboard title:', title);

  // 5. Clean up
  await browser.close();
  await fetch(
    `https://api.adscrawl.net/cdp/sessions/${session.sessionId}`,
    { method: 'DELETE', headers: { 'x-api-key': process.env.ADSCRAWL_API_KEY! } }
  );
}

main();
```

```python Python (Playwright) theme={null}
import os
from playwright.sync_api import sync_playwright
import requests

# 1. Create a CDP session
session_res = requests.post(
    'https://api.adscrawl.net/cdp/sessions',
    headers={'x-api-key': os.environ['ADSCRAWL_API_KEY']},
    json={
        'idleTimeoutMs': 300000,
        'maxSessionMs': 1800000,
        'browserSettings': {
            'countryCode': 'US',
            'userAgentMode': 'random',
            'viewport': {'width': 1440, 'height': 900},
        },
    },
)
session = session_res.json()

# 2. Connect and automate
with sync_playwright() as p:
    browser = p.chromium.connect_over_cdp(session['cdpBaseUrl'])
    page = browser.new_page()

    # 3. Navigate and interact
    page.goto('https://example.com/login')
    page.fill('#email', 'user@example.com')
    page.fill('#password', 'secret')
    page.click('button[type=submit]')
    page.wait_for_selector('.dashboard')

    # 4. Collect results
    print('Dashboard title:', page.title())

    # 5. Close browser
    browser.close()

# Delete the session
requests.delete(
    f"https://api.adscrawl.net/cdp/sessions/{session['sessionId']}",
    headers={'x-api-key': os.environ['ADSCRAWL_API_KEY']},
)
```

## Session limits and 429 errors

Each API key has a maximum number of concurrent CDP sessions. Attempting to create a new session when the limit is reached returns `429`:

```json theme={null}
{
  "error": "CDP sessions per API key limit reached"
}
```

List your active sessions to inspect what's running before creating new ones:

```bash theme={null}
curl -sS "https://api.adscrawl.net/cdp/sessions" \
  -H "x-api-key: $ADSCRAWL_API_KEY"
```

Delete any sessions you no longer need, then retry.

## Session timeouts

> [!TIP]
>
> Set `idleTimeoutMs` to automatically terminate the session after a period of inactivity. This prevents runaway costs if your automation crashes before reaching the cleanup step. Values above the server cap are automatically clamped.

Use `maxSessionMs` as a hard wall-clock limit for long-running automations. Both timeouts are enforced server-side — no client-side keepalive is required.

## Data token security

> [!WARNING]
>
> The `cdpBaseUrl` contains a data token that grants direct access to your browser session's CDP WebSocket. Treat it like a credential: never log it, embed it in client-side code, or include it in URLs that appear in browser history. Keep it server-side only.

## browserSettings reference

> [accordion] browserSettings fields
>
> **countryCode** `string`
>
> Managed proxy region. `"GLOBAL"` picks from 15 popular regions. A two-letter code (e.g. `"US"`, `"DE"`) prefers a trusted proxy with dynamic fallback. Cannot be combined with `proxy`.
>
> **viewport** `object`
>
> Browser window size, e.g. `{ "width": 1440, "height": 900 }`.
>
> **userAgentMode** `"random" | "custom"`
>
> Set to `"random"` for a server-selected realistic User-Agent string.
>
> **userAgentOs** `"windows" | "macos"`
>
> Operating system for random User-Agent selection. Defaults to `"windows"`.
>
> **locale** `string`
>
> Browser locale, such as `"en-US"`.
>
> **timezoneId** `string`
>
> IANA timezone ID, such as `"America/New_York"`.
>
> **cookies** `cookies[]`
>
> Cookies injected into the browser context before any navigation.
>
> **fingerprint** `fingerprint`
>
> Browser fingerprint settings. When omitted, `canvas` and `webGlImage` default to `real`; other signals use a coherent randomised profile.
>
> **proxy** `proxy`
>
> Custom proxy configuration. Cannot be combined with `countryCode`.


# Python Website Scraper Examples
Source: https://docs.adscrawl.net/guides/scraper-examples

Run and customize AdsCrawl's Python scraper examples for search, products, reviews, real estate, and other websites.

[AdsCrawl Scrapers](https://github.com/AdsCrawl/adscrawl-scrapers) is a collection of Python examples for 81 websites. Each site directory contains its parser, a runnable script, live tests, and historical sample output. Use the examples as a starting point for a specific website, or reuse the shared remote browser client for your own parser.

> [card] [Browse the source](https://github.com/AdsCrawl/adscrawl-scrapers)
>
> Find site modules, run scripts, and the complete scraper directory.

> [card] [Python SDK](/sdks/python)
>
> Learn about authentication, API methods, and remote CDP sessions.

## Install and configure

You need Git, **Python 3.10 or later**, and an [AdsCrawl API key](https://app.adscrawl.net/dashboard/keys/). The scraper repository has a higher Python requirement than the standalone SDK.

Run these commands from a terminal:

```bash theme={null}
git clone https://github.com/AdsCrawl/adscrawl-scrapers.git
cd adscrawl-scrapers
python -m venv .venv
```

> [tab] macOS / Linux
>
> ```bash theme={null}
> source .venv/bin/activate
> python -m pip install -e ".[dev]"
> export ADSCRAWL_API_KEY="your_api_key"
> ```

> [tab] Windows PowerShell
>
> ```powershell theme={null}
> .\.venv\Scripts\Activate.ps1
> python -m pip install -e ".[dev]"
> $env:ADSCRAWL_API_KEY = "your_api_key"
> ```

The editable install provides `adscrawl_scrapers`, the AdsCrawl SDK, Playwright, parsers, and development test dependencies. These examples connect to **remote Chromium browsers**; you do not need to run `playwright install chromium`.

> [!NOTE]
>
> Running a scraper or a site's live tests uses AdsCrawl credits. Review the input URLs, dates, and page limits in `run.py` before running it. Keep API keys in environment variables and out of source control.

## Example: Bing search and related keywords

From the repository root, run the included script:

```bash theme={null}
cd bing-scraper
python run.py
```

The script calls these functions from `bing.py`:

| Function | Input | Output |
| - | - | - |
| `scrape_search(query, max_pages)` | Search query and a positive integer page limit | A list of search result dictionaries |
| `scrape_keywords(query)` | Search query | A list of related keyword strings |

The included `run.py` searches for `web scraping emails` across up to three pages and writes `results/serps.json` and `results/keywords.json` inside `bing-scraper/`.

To customize the query and start with one page, save this as `bing-scraper/custom.py` and run `python custom.py` from that directory:

```python theme={null}
import json
from pathlib import Path

import bing
from adscrawl_scrapers import run


async def main():
    query = "web scraping emails"
    results = await bing.scrape_search(query=query, max_pages=1)
    keywords = await bing.scrape_keywords(query=query)

    output = Path(__file__).parent / "custom-results"
    output.mkdir(exist_ok=True)
    (output / "search.json").write_text(
        json.dumps(results, ensure_ascii=False, indent=2), encoding="utf-8"
    )
    (output / "keywords.json").write_text(
        json.dumps(keywords, ensure_ascii=False, indent=2), encoding="utf-8"
    )
    print(f"Saved {len(results)} results and {len(keywords)} keywords")


if __name__ == "__main__":
    run(main())
```

Search results include `position`, `title`, `url`, `origin`, `domain`, `description`, and `date`; some values can be absent or null. Bing may return no related keywords for a particular query. The module explicitly uses `countryCode="GB"`, so the default country environment variable does not override its route.

## Example: ChatGPT answer as Markdown

From the repository root:

```bash theme={null}
cd chatgpt-scraper
python run.py
```

The included script calls `chatgpt.scrape_conversation(prompt)` and saves the returned Markdown to `chatgpt-scraper/results/conversation.md`. Edit the prompt in `run.py` to change the question.

For a small standalone example, save this as `chatgpt-scraper/custom.py` and run `python custom.py` from that directory:

```python theme={null}
from pathlib import Path

import chatgpt
from adscrawl_scrapers import run


async def main():
    markdown = await chatgpt.scrape_conversation(
        "What is the capital of France?"
    )
    Path(__file__).with_name("answer.md").write_text(
        markdown, encoding="utf-8"
    )


if __name__ == "__main__":
    run(main())
```

This example operates the ChatGPT website in a remote browser and converts page content to Markdown. The returned text can include surrounding page content. Availability depends on the website's current UI and access requirements.

## Choose another website

Open a site's directory, inspect its `run.py`, update the inputs, and run it from that directory. Function signatures and output schemas differ by site; the modules are local files such as `bing.py`, rather than submodules of the installed `adscrawl` SDK.

| Use case | Example directories | Typical data |
| - | - | - |
| Search and AI | [Bing](https://github.com/AdsCrawl/adscrawl-scrapers/tree/main/bing-scraper), [Google](https://github.com/AdsCrawl/adscrawl-scrapers/tree/main/google-scraper), [ChatGPT](https://github.com/AdsCrawl/adscrawl-scrapers/tree/main/chatgpt-scraper) | Search results, related keywords, page answers |
| Products and commerce | [Shopify](https://github.com/AdsCrawl/adscrawl-scrapers/tree/main/shopify-scraper), [Idealo](https://github.com/AdsCrawl/adscrawl-scrapers/tree/main/idealo-scraper), [Amazon](https://github.com/AdsCrawl/adscrawl-scrapers/tree/main/amazon-scraper) | Catalogs, products, prices, reviews |
| Social content | [Reddit](https://github.com/AdsCrawl/adscrawl-scrapers/tree/main/reddit-scraper), [Twitter / X](https://github.com/AdsCrawl/adscrawl-scrapers/tree/main/twitter-scraper), [Instagram](https://github.com/AdsCrawl/adscrawl-scrapers/tree/main/instagram-scraper) | Public posts, profiles, comments |
| Real estate | [Zillow](https://github.com/AdsCrawl/adscrawl-scrapers/tree/main/zillow-scraper), [Rightmove](https://github.com/AdsCrawl/adscrawl-scrapers/tree/main/rightmove-scraper), [Imovelweb](https://github.com/AdsCrawl/adscrawl-scrapers/tree/main/imovelweb-scraper) | Search results and property details |
| Apps, media, and events | [App Store](https://github.com/AdsCrawl/adscrawl-scrapers/tree/main/app-store-scraper), [IMDb](https://github.com/AdsCrawl/adscrawl-scrapers/tree/main/imdb-scraper), [Ticketmaster](https://github.com/AdsCrawl/adscrawl-scrapers/tree/main/ticketmaster-scraper) | App reviews, titles, charts, event discovery |
| Travel and jobs | [Booking.com](https://github.com/AdsCrawl/adscrawl-scrapers/tree/main/bookingcom-scraper), [Google Flights](https://github.com/AdsCrawl/adscrawl-scrapers/tree/main/google-flights-scraper), [Indeed](https://github.com/AdsCrawl/adscrawl-scrapers/tree/main/indeed-scraper) | Hotels, flight searches, job listings |

These links describe the examples present in the repository. Check the verification report for the status of the functions you plan to use. See the [complete list of 81 sites](https://github.com/AdsCrawl/adscrawl-scrapers#scrapers) and [machine-readable function catalog](https://github.com/AdsCrawl/adscrawl-scrapers/blob/main/docs/catalog.json).

## Concurrency, region, and sessions

| Setting | Default | Behavior |
| - | - | - |
| `ADSCRAWL_API_KEY` | Required | API key used by the SDK |
| `ADSCRAWL_CONCURRENCY` | `2` | Maximum simultaneous browser sessions per shared scraper client; use a positive integer within your account's capacity |
| `ADSCRAWL_DEFAULT_COUNTRY` | `US` | Route used when a request has neither an explicit `countryCode` nor a custom `proxy` |

For example, on macOS or Linux:

```bash theme={null}
export ADSCRAWL_CONCURRENCY="1"
export ADSCRAWL_DEFAULT_COUNTRY="US"
```

Set these variables before starting Python. In PowerShell, use `$env:ADSCRAWL_CONCURRENCY = "1"` and `$env:ADSCRAWL_DEFAULT_COUNTRY = "US"`.

A site's explicit `countryCode` takes precedence. A custom `proxy` cannot be combined with `countryCode`. Multiple clients or processes can consume more sessions than one client's limit, so consider their combined usage.

Most examples use `AsyncAdsCrawl.cdp.create`, connect Playwright over CDP, capture HTML and network responses, and release sessions with `cdp.close`. Named sessions reuse cookies, storage, and the browser connection within a site operation. Standalone scripts use `adscrawl_scrapers.run(...)` to close shared clients on exit. Air France uses the synchronous SDK; Google AI Mode uses `spa.extract`.

## Build your own parser

After installing the repository, save this as a Python script and run it with your API key configured:

```python theme={null}
from adscrawl_scrapers import Request, ScraperClient, run


async def main():
    async with ScraperClient(concurrency=1) as client:
        response = await client.fetch(
            Request("https://example.com", countryCode="US")
        )
        print(response.selector.css("title::text").get())


if __name__ == "__main__":
    run(main())
```

Use `response.content` for the page body, `response.selector` for CSS/XPath extraction, and `response.network` for captured XHR/fetch responses. For a JSON response body, use `response.json()`. The asynchronous context manager closes the client's remote sessions on exit.

## Verify results

The repository's [dated live report](https://github.com/AdsCrawl/adscrawl-scrapers/blob/main/docs/live-verification/2026-10-27.md) records **suites across 81 sites on October 27, 2026**. For that audit, Bing returned 30 search results and 8 related keywords; the ChatGPT test returned an answer containing “Paris.” Those counts describe the recorded tests, not guaranteed output from the snippets above.

The fully passed suites were App Store, Bing, ChatGPT, Fashionphile, Idealo, IMDb, Imovelweb, Reddit, Rightmove, Shopify, Similarweb, Ticketmaster, Twitter / X, Vestiaire Collective, and Zillow. Passing applies to the existing tested functions and inputs on that date. Some individual site READMEs still contain earlier verification notes; use the dated report for this audit's evidence and scope.

Files already in `results/` are historical samples. Inspect newly generated output and required fields before using the data; an empty result does not establish that a scraper worked.

From the repository root, run offline checks without an API key or credit usage:

```bash theme={null}
python -m pytest tests -q
python scripts/check_catalog.py
```

To test Bing against the live website with your configured API key:

```bash theme={null}
python -m pytest bing-scraper/test.py -v
```

The last command uses credits. Default CI runs offline checks; a selected scraper's live workflow is triggered manually.

## Troubleshooting

| Symptom | What to check |
| - | - |
| `No module named adscrawl_scrapers` | Activate the virtual environment and run `python -m pip install -e ".[dev]"` from the repository root. |
| `No module named bing` or another site module | Put your script in the matching site directory alongside its module, then run it there. |
| Missing or invalid API key | Set `ADSCRAWL_API_KEY` in the same terminal before running the script. |
| API capacity error or `SESSION_LIMIT` | Reduce concurrency, account for other clients, and inspect active CDP sessions. Close named sessions you no longer need. |
| Target HTTP 403/429, login page, or empty output | Review the dated report, target URL, access requirements, and country settings; check the returned content before trusting parsed data. |
| `TIMEOUT` or `SELECTOR_NOT_FOUND` | Check whether the site has changed its DOM or loading behavior. Review the site's selectors and bounded wait settings. |
| `CLEANUP_FAILED` | Inspect active CDP sessions and close the sessions belonging to this run before retrying. |

The shared transport does not automatically retry billable requests. Some site modules contain their own retry loops; inspect the module before increasing the workload. See [credits and limits](/credits-and-limits), [remote CDP](/concepts/remote-cdp), and the repository's [transport notes](https://github.com/AdsCrawl/adscrawl-scrapers/blob/main/docs/MIGRATION.md).


# Capture Full-Page and Element Screenshots with AdsCrawl
Source: https://docs.adscrawl.net/guides/screenshots

Use POST /screenshot to capture a full-page or single-element PNG from any URL, with proxy routing, custom viewports, and locale support.

Use `POST /screenshot` to capture a PNG image of any web page without managing a browser. AdsCrawl launches a real Chromium instance, routes the request through a residential proxy, waits for the page to load, and streams the image directly back as a binary PNG response. Screenshots are useful for visual monitoring, AI-powered workflows, research archiving, and automated reporting.

## Full-page screenshot

Set `fullPage: true` to capture the entire scrollable page at the specified viewport width. Use `--output` with cURL to write the binary response directly to a file.

```bash theme={null}
curl -sS -X POST "https://api.adscrawl.net/screenshot" \
  -H "content-type: application/json" \
  -H "x-api-key: $ADSCRAWL_API_KEY" \
  -d '{
    "url": "https://example.com",
    "viewport": { "width": 1440, "height": 900 },
    "fullPage": true,
    "waitUntil": "load"
  }' \
  --output page.png
```

## Element screenshot

Provide a `selector` to capture only the first matching element. The viewport still controls page layout — only the matched element is cropped from the rendered page.

```bash theme={null}
curl -sS -X POST "https://api.adscrawl.net/screenshot" \
  -H "content-type: application/json" \
  -H "x-api-key: $ADSCRAWL_API_KEY" \
  -d '{
    "url": "https://example.com",
    "selector": ".hero-banner",
    "fullPage": false,
    "waitUntil": "load"
  }' \
  --output banner.png
```

> [!NOTE]
>
> If the `selector` you provide doesn't match any element on the page, the API returns `422 CONTENT_SELECTOR_NOT_FOUND`. Check your selector against the live rendered DOM — some elements only appear after JavaScript execution completes.

## waitUntil options

The `waitUntil` field controls when AdsCrawl considers the page ready to capture:

| Value | When it fires | Use when |
| - | - | - |
| `load` *(default)* | After `window.load`, including images and stylesheets | You need the fully styled page |
| `domcontentloaded` | After HTML is parsed, before secondary resources | You want speed and don't need images |
| `networkidle` | After 500 ms with no network activity | The page makes async API calls to fill in content |

> [!WARNING]
>
> `networkidle` can time out on pages with persistent connections, analytics beacons, or lazy-loaded infinite scroll content. Prefer `load` for most screenshot use cases.

## Tips

> [!TIP]
>
> Use `countryCode` to capture geo-specific page variants. Set `"countryCode": "US"` to see the US version of a site, or `"countryCode": "GLOBAL"` to use a randomly selected residential exit node across 15 popular regions.

> [!TIP]
>
> Set `locale` and `timezoneId` together to capture localised pages accurately. For example, `"locale": "zh-CN"` and `"timezoneId": "Asia/Shanghai"` render the page as a user in China would see it.

## Use cases

> [card] Visual monitoring
>
> Schedule periodic screenshots to detect layout regressions or content changes on competitor sites.

> [card] AI workflows
>
> Feed screenshots into multimodal LLMs for visual page understanding, UI audits, or content classification.

> [card] Research and archiving
>
> Capture a timestamped visual record of pages for compliance, journalism, or market research.

## Full request body reference

> [accordion] All request fields
>
> **url** `string` *required*
>
> Target page URL. Only ports 80 and 443 are supported.
>
> **viewport** `object`
>
> Viewport dimensions, e.g. `{ "width": 1440, "height": 900 }`. Controls page layout width; the captured height extends to the full page when `fullPage` is true.
>
> **fullPage** `boolean`
>
> Capture the entire scrollable page. Defaults to `true`. When `selector` is set, only the matched element is captured.
>
> **selector** `string`
>
> CSS selector for a single element to capture. Returns `422 CONTENT_SELECTOR_NOT_FOUND` if no element matches.
>
> **waitUntil** `"load" | "domcontentloaded" | "networkidle"`
>
> Navigation wait condition. Defaults to `"load"`.
>
> **countryCode** `string`
>
> Managed proxy region. `"GLOBAL"` picks a dynamic exit from 15 regions. A two-letter code (e.g. `"US"`) prefers a trusted proxy. Cannot be combined with `proxy`.
>
> **locale** `string`
>
> Browser locale, such as `"en-US"` or `"zh-CN"`.
>
> **timezoneId** `string`
>
> IANA timezone ID, such as `"Asia/Shanghai"` or `"America/New_York"`.
>
> **cookies** `cookies[]`
>
> Cookie list injected before navigation, useful for capturing authenticated pages.
>
> **userAgentMode** `"random" | "custom"`
>
> Set to `"random"` to have AdsCrawl select a realistic User-Agent. Defaults to `"random"` when no User-Agent is provided.
>
> **timeoutMs** `number`
>
> Navigation timeout in milliseconds. Must be positive and no greater than 3,600,000.


# AdsCrawl: Browser Automation API for Scraping the Web
Source: https://docs.adscrawl.net/introduction

AdsCrawl is a browser automation API for developers who need rendered HTML, structured data, screenshots, and full CDP sessions via residential proxies.

AdsCrawl gives your application a full browser environment — backed by residential proxies — without the overhead of managing infrastructure. Send a request with a URL, and AdsCrawl spins up a real Chromium browser, renders the page, and returns exactly what you asked for: raw HTML, clean Markdown, structured JSON, a PNG screenshot, or a live CDP connection for multi-step automation. Whether you're building a scraper, a monitoring tool, a data pipeline, or an AI workflow, AdsCrawl handles the rendering, proxy routing, and fingerprinting so you can focus on your data.

## What you can do

AdsCrawl exposes five core capabilities through a single base URL (`https://api.adscrawl.net`):

* **HTML extraction** — Fetch fully rendered HTML from any URL, or extract clean Markdown and structured article JSON using the built-in Readability parser.
* **Structured data extraction** — Pull specific fields from dynamic single-page apps using CSS selectors or intercepted network responses, with optional pre-built site templates for popular targets.
* **Screenshots** — Capture a full-page PNG or isolate a single element with a CSS selector.
* **Remote CDP** — Create a dedicated Chromium session and connect Playwright or Puppeteer over the Chrome DevTools Protocol for scripted, multi-step browser automation.
* **Cloud Browsers** — Launch a persistent browser profile with your own proxy and cookies, view it live in the interactive viewer, stop it, and reuse the saved profile on your next run.

## Explore the docs

> [card] [Quick Start](/quickstart)
>
> Create an account, get your API key, and make your first request in minutes.

> [card] [Authentication](/authentication)
>
> Learn how to pass your API key and understand session tokens and data tokens.

> [card] [Browser Tasks](/concepts/browser-tasks)
>
> HTML extraction, structured data, and screenshots — all synchronous, all proxied.

> [card] [Remote CDP](/concepts/remote-cdp)
>
> Connect Playwright or Puppeteer to a remote browser over CDP for full automation.

> [card] [Cloud Browsers](/concepts/cloud-browsers)
>
> Persistent profiles, live viewer, proxy-per-session, and reusable state.

> [card] [API Reference](/api-reference/html)
>
> Complete endpoint reference with request and response schemas.

## How it works

Every AdsCrawl request follows the same pattern regardless of which API you use:

1. **Send a request** — Your server sends an HTTP request with your API key and the target URL (plus any options such as `waitUntil`, `contentMode`, or `countryCode`).
2. **Browser renders the page** — AdsCrawl launches a Chromium instance, routes it through a residential proxy, injects any cookies or fingerprint settings you specified, and waits for the page to reach the load state you requested.
3. **Get back your data** — The rendered HTML, Markdown, JSON, PNG binary, or CDP session URL is returned directly in the HTTP response. There is no polling or webhook step for the synchronous APIs.

For stateful workflows — scripted navigation, form submission, multi-step login flows — use the Remote CDP or Cloud Browser APIs, which keep the browser alive across multiple requests.

> [!NOTE]
>
> Residential proxies are built into every request. You can route through a specific two-letter country code, use `GLOBAL` for dynamic region selection across 15 popular regions, or supply your own custom proxy. There is no separate proxy subscription required.


# AdsCrawl Pricing: Free, Hobby, Starter, and Pro Plans
Source: https://docs.adscrawl.net/pricing

Compare Free, Hobby, Starter, and Pro plans. Choose the credit allowance, Cloud Browser profiles, and API keys that fit your automation needs.

AdsCrawl uses a credit-based pricing model: every plan comes with a monthly credit allowance (or a one-time grant on Free) that you spend as you use the API. Browser task requests each consume one credit, and Cloud Browsers bill at one credit per started minute. Pick the plan that matches your volume — start on Free to explore the APIs, then move to a paid tier when you need more credits, more saved browser profiles, or more API keys.

> [card] Free
>
> **\$0** — no monthly fee
>
> * **500** one-time credits
> * **1** Cloud Browser profile *(save now; upgrade to launch)*
> * **1** API key
>
> [Start free →](https://app.adscrawl.net/register/)

> [card] Hobby
>
> **\$9 / month**
>
> * **3,000** credits per month
> * **20** Cloud Browser profiles
> * **3** API keys
>
> [Choose Hobby →](https://app.adscrawl.net/register/)

> [card] Starter
>
> **\$49 / month** ⭐ Recommended
>
> * **100,000** credits per month
> * **50** Cloud Browser profiles
> * **10** API keys
>
> [Choose Starter →](https://app.adscrawl.net/register/)

> [card] Pro
>
> **\$199 / month**
>
> * **500,000** credits per month
> * **200** Cloud Browser profiles
> * **30** API keys
>
> [Choose Pro →](https://app.adscrawl.net/register/)

> [!NOTE]
>
> The Free plan lets you **save** one Cloud Browser profile, but you need an active paid plan to **launch** it. Upgrade at any time from the [dashboard](https://app.adscrawl.net/dashboard/).

> [!NOTE]
>
> **Annual billing coming soon.** Monthly billing is currently the only available cycle. Check the dashboard for updates on annual pricing.

## Frequently Asked Questions

> [accordion] Do credits renew every month?
>
> Yes, on all paid plans. Credits reset at the start of each billing cycle — 3,000 on Hobby, 100,000 on Starter, and 500,000 on Pro. The Free plan's 500 credits are a **one-time grant** and do not renew.

> [accordion] Can I run a Cloud Browser on the Free plan?
>
> You can **save** one Cloud Browser profile on the Free plan, but you need an active paid plan to **launch** it. Concurrent run limits for paid plans are configured on your account. Upgrade to Hobby or above to start running browsers.

> [accordion] What counts as one credit?
>
> Each browser task request (POST `/html`, POST `/screenshot`, POST `/spa-extract`) consumes **1 credit** when the request passes validation and before execution begins. Cloud Browsers consume **1 credit per started minute**, counted from a successful start until the browser stops. Credits are not refunded if a request fails after consumption has already occurred.

> [accordion] How many API keys are included?
>
> Each plan includes a different API key allowance:
>
> | Plan | API Keys |
> | - | - |
> | Free | 1 |
> | Hobby | 3 |
> | Starter | 10 |
> | Pro | 30 |
>
> Keep your API keys on a trusted server or in a secret manager — never expose them in client-side code, URLs, or logs.

> [accordion] Where do I see my plan and remaining credits?
>
> Sign in to the [AdsCrawl dashboard](https://app.adscrawl.net/dashboard/) and open **Billing & Usage**. You'll see your current plan, credit balance, and account limits including Cloud Browser profile capacity and concurrent run allowance.

> [accordion] What happens if I run out of credits?
>
> Once your credit balance reaches zero, API requests return a **402 INSUFFICIENT\_CREDITS** error. Top up your balance or upgrade to a higher plan from the [Billing & Usage](https://app.adscrawl.net/dashboard/) section of the dashboard. Requests are not queued — they fail immediately until credits are available.


# Quick Start: Make Your First AdsCrawl API Request Today
Source: https://docs.adscrawl.net/quickstart

Create a free AdsCrawl account, grab your API key from the dashboard, and make your first rendered HTML or Markdown request in under five minutes.

AdsCrawl is designed to be usable from your first request. This guide walks you from account creation to a working API call — no configuration files, no infrastructure setup, and no proxy accounts to manage separately. By the end, you'll have a rendered page response in your terminal and a clear path to whichever API fits your use case.

**1. Create a free account**

Go to [app.adscrawl.net/register](https://app.adscrawl.net/register/) and sign up. The Free plan gives you 500 credits with no credit card required — enough to explore HTML extraction, screenshots, and short CDP sessions.

**2. Copy your API key**

After signing in, open the [AdsCrawl dashboard](https://app.adscrawl.net/dashboard/) and copy your API key. Store it as an environment variable on your server or in a secret manager.

```bash theme={null}
export ADSCRAWL_API_KEY="your-api-key"
```

> [!WARNING]
>
> Never paste your API key into client-side JavaScript, commit it to source control, or include it in a public URL. Keep it on your server only.

**3. Make your first request**

Use `POST /html` to fetch a rendered page. This endpoint is synchronous — AdsCrawl launches a real browser, loads the URL, and streams the result back in the HTTP response.

Set `contentMode` to control what you get back:

* `"html"` — full rendered page HTML (default)
* `"markdown"` — clean, readable Markdown extracted by the Readability parser
* `"json"` — structured article object with title, byline, excerpt, and body

```bash cURL theme={null}
export ADSCRAWL_API_KEY="your-api-key"

curl --fail-with-body -sS -X POST "https://api.adscrawl.net/html" \
  -H "content-type: application/json" \
  -H "x-api-key: $ADSCRAWL_API_KEY" \
  -d '{
    "url": "https://example.com/article",
    "contentMode": "markdown",
    "waitUntil": "domcontentloaded"
  }'
```

```javascript JavaScript (fetch) theme={null}
const response = await fetch("https://api.adscrawl.net/html", {
  method: "POST",
  headers: {
    "content-type": "application/json",
    "x-api-key": process.env.ADSCRAWL_API_KEY,
  },
  body: JSON.stringify({
    url: "https://example.com/article",
    contentMode: "markdown",
    waitUntil: "domcontentloaded",
  }),
});

const markdown = await response.text();
console.log(markdown);
```

```python Python theme={null}
import os
import requests

response = requests.post(
    "https://api.adscrawl.net/html",
    headers={
        "content-type": "application/json",
        "x-api-key": os.environ["ADSCRAWL_API_KEY"],
    },
    json={
        "url": "https://example.com/article",
        "contentMode": "markdown",
        "waitUntil": "domcontentloaded",
    },
)

print(response.text)
```

Use `"waitUntil": "domcontentloaded"` for most HTML and Markdown extraction — it's faster and avoids waiting on analytics or lazy-loaded resources. Switch to `"load"` or `"networkidle"` when the content you need depends on secondary resources or deferred JavaScript.

**4. Check the response**

A successful request returns HTTP `200`. The response body depends on `contentMode`:

> [tab] markdown (text/markdown)
>
> ```markdown theme={null}
> # Example Article
>
> Readable body content extracted from the page.
>
> - Key point one
> - Key point two
> ```

> [tab] json (application/json)
>
> ```json theme={null}
> {
>   "title": "Example Article",
>   "byline": "Jane Smith",
>   "excerpt": "A concise summary of the article.",
>   "siteName": "Example",
>   "lang": "en",
>   "dir": null,
>   "content": "<div><p>Readable body...</p></div>",
>   "textContent": "Readable body...",
>   "length": 2487,
>   "publishedTime": null
> }
> ```

> [tab] html (text/html)
>
> ```html theme={null}
> <!DOCTYPE html>
> <html lang="en">
>   <head><title>Example Article</title></head>
>   <body>
>     <!-- Full rendered page HTML -->
>   </body>
> </html>
> ```

If you receive a `402`, your account is out of credits — check your balance in the dashboard. A `401` means the API key is missing or incorrect.

## What to explore next

AdsCrawl has five APIs. Start with the one that fits your workflow:

> [card] [Browser Tasks](/concepts/browser-tasks)
>
> HTML, Markdown, JSON, screenshots, and structured field extraction from SPAs.

> [card] [Remote CDP](/concepts/remote-cdp)
>
> Connect Playwright or Puppeteer to a remote browser for scripted automation.

> [card] [Cloud Browsers](/concepts/cloud-browsers)
>
> Persistent browser profiles with a live viewer, saved cookies, and reusable state.

> [card] [API Reference](/api-reference/html)
>
> Complete request and response schemas for every endpoint.

> [!TIP]
>
> If you'd prefer to let your coding agent handle setup, paste this prompt into your agent:
>
> ```text theme={null}
> Set up AdsCrawl in this codebase. Use https://api.adscrawl.net/auth.md to help me
> create an account and authorize an API key, then follow
> https://api.adscrawl.net/docs/agent-quickstart.md to complete the integration.
> ```
>
> The agent will walk you through sign-up and integration. You approve the API key request in your browser.

Once you've made your first successful request, read the [Authentication](/authentication) page to understand the full credential model — especially if you plan to use Cloud Browsers or CDP sessions.


# AdsCrawl Codex Plugin
Source: https://docs.adscrawl.net/sdks/codex

Use the AdsCrawl Codex plugin to render pages, and capture full-page screenshots from OpenAI Codex. Requires Python 3.9+ and an API key.

The AdsCrawl Codex plugin adds browser-rendered page reading and screenshots to [OpenAI Codex](https://openai.com/codex/). It bundles an `adscrawl-browser` skill and a Python 3.9+ command that uses only the standard library. It calls the AdsCrawl API using your `ADSCRAWL_API_KEY` and does not bundle credentials.

## Features

* Render a URL as Markdown, HTML, or structured article JSON.
* Capture a full-page or viewport PNG screenshot.
* Select a managed country route and navigation wait strategy.
* Use AdsCrawl CDP sessions for interactive browser work through your project's browser automation library.

## Requirements

* Python 3.9 or later.
* An [AdsCrawl account](https://app.adscrawl.net/register/) with an API key.

## Install

Install the plugin from the public GitHub repository:

```bash theme={null}
codex plugin marketplace add AdsCrawl/adscrawl-codex
codex plugin add adscrawl@adscrawl
```

Start a new Codex task after installation so the bundled skill is loaded. To add a local checkout while developing:

```bash theme={null}
codex plugin marketplace add /path/to/adscrawl-codex
codex plugin add adscrawl@adscrawl
```

> [card] [adscrawl-codex on GitHub](https://github.com/AdsCrawl/adscrawl-codex)
>
> View source and releases.

## Authenticate

After installing the plugin, ask Codex to connect your AdsCrawl account, or run the auth command directly:

```bash theme={null}
python3 plugins/adscrawl/scripts/adscrawl.py auth
```

The command asks for your account email and a project name, then prints a link for you to approve. It never asks for your password or prints the resulting API key.

Alternatively, set `ADSCRAWL_API_KEY` in your environment. The plugin will reuse an existing environment variable.

The auth flow saves your key privately at `~/.config/adscrawl/codex.env` with `0600` permissions.

## Direct commands

The skill invokes `plugins/adscrawl/scripts/adscrawl.py`. You can also run it directly:

### Render a page

```bash theme={null}
python3 plugins/adscrawl/scripts/adscrawl.py render \
  --url 'https://www.adscrawl.net/' --format markdown --output page.md
```

> [!NOTE]
>
> `--format` accepts `markdown`, `html`, or `json` (structured article).

### Capture a screenshot

```bash theme={null}
python3 plugins/adscrawl/scripts/adscrawl.py screenshot \
  --url 'https://www.adscrawl.net/' --output page.png
```

Run `python3 plugins/adscrawl/scripts/adscrawl.py <command> --help` for the available options. API failures are not retried automatically.

## CDP sessions

For interactive browser work (clicking, filling forms, multi-step flows), use AdsCrawl CDP sessions through your project's browser automation library. See [Remote CDP browsers](/concepts/remote-cdp) and [Playwright automation](/guides/playwright-automation).

> [!NOTE]
>
> Plugin requests can consume AdsCrawl credits. See [Credits and limits](/credits-and-limits).


# Go SDK for AdsCrawl
Source: https://docs.adscrawl.net/sdks/go

Install and use the AdsCrawl Go SDK to fetch rendered content, screenshots, structured data, and manage remote CDP sessions. Go 1.21+, context-aware, zero runtime dependencies.

The AdsCrawl Go SDK works with Go 1.21 and later. It uses only the Go standard library for HTTP, supports `context.Context` cancellation and deadlines, and provides typed request and response models.

> [!NOTE]
>
> Keep your API key in server-side code. Never expose it in client-side applications.

## Install

```bash theme={null}
go get github.com/AdsCrawl/adscrawl-go@v0.1.0
```

> [card] [adscrawl-go on GitHub](https://github.com/AdsCrawl/adscrawl-go)
>
> View source, runnable examples, and releases.

## Authenticate and make your first request

[Create an API key](https://app.adscrawl.net/register/?utm_source=go\&utm_medium=sdk\&utm_campaign=adscrawl-go) in the dashboard and set `ADSCRAWL_API_KEY` in your server environment. You can also pass the key to `adscrawl.NewClient(adscrawl.Config{APIKey: "..."})`.

```go theme={null}
package main

import (
    "context"
    "fmt"
    "log"

    adscrawl "github.com/AdsCrawl/adscrawl-go"
)

func main() {
    client, err := adscrawl.NewClient(adscrawl.Config{})
    if err != nil {
        log.Fatal(err)
    }
    markdown, err := client.Markdown(context.Background(), adscrawl.ContentOptions{
        PageOptions: adscrawl.PageOptions{
            URL:       "https://www.adscrawl.net",
            WaitUntil: "domcontentloaded",
        },
    })
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(markdown)
}
```

## Rendered content and screenshots

`client.HTML(ctx, options)` returns HTML; `Markdown` returns Markdown; `Article` returns a structured `Article`; `Screenshot` returns PNG bytes. Page options include `Viewport`, `Locale`, `Cookies`, `Proxy`, managed `CountryCode`, `Fingerprint`, server-side `TimeoutMS`, and navigation `WaitUntil`.

```go theme={null}
package main

import (
    "context"
    "fmt"
    "log"
    "os"

    adscrawl "github.com/AdsCrawl/adscrawl-go"
)

func main() {
    client, err := adscrawl.NewClient(adscrawl.Config{})
    if err != nil {
        log.Fatal(err)
    }

    html, err := client.HTML(context.Background(), adscrawl.ContentOptions{
        PageOptions: adscrawl.PageOptions{URL: "https://www.adscrawl.net"},
    })
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(html)

    article, err := client.Article(context.Background(), adscrawl.ContentOptions{
        PageOptions: adscrawl.PageOptions{URL: "https://www.adscrawl.net"},
    })
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(article.Title, article.TextContent)

    png, err := client.Screenshot(context.Background(), adscrawl.ScreenshotOptions{
        PageOptions: adscrawl.PageOptions{
            BrowserSettings: adscrawl.BrowserSettings{
                Viewport: &adscrawl.Viewport{Width: 1440, Height: 900},
            },
            URL:       "https://www.adscrawl.net",
            WaitUntil: "load",
        },
        FullPage: true,
    })
    if err != nil {
        log.Fatal(err)
    }
    if err := os.WriteFile("page.png", png, 0600); err != nil {
        log.Fatal(err)
    }
}
```

A custom `Proxy` and `CountryCode` cannot be combined.

## Proxy and fingerprint

```go theme={null}
package main

import (
    "context"
    "log"
    "os"

    adscrawl "github.com/AdsCrawl/adscrawl-go"
)

func main() {
    client, err := adscrawl.NewClient(adscrawl.Config{})
    if err != nil {
        log.Fatal(err)
    }

    routing := adscrawl.BrowserSettings{CountryCode: "GLOBAL"}
    if server := os.Getenv("ADSCRAWL_PROXY_SERVER"); server != "" {
        username, password := os.Getenv("ADSCRAWL_PROXY_USERNAME"), os.Getenv("ADSCRAWL_PROXY_PASSWORD")
        if (username == "") != (password == "") {
            log.Fatal("set both proxy username and password")
        }
        routing = adscrawl.BrowserSettings{
            Proxy: &adscrawl.Proxy{Server: server, Username: username, Password: password},
        }
    } else if os.Getenv("ADSCRAWL_PROXY_USERNAME") != "" || os.Getenv("ADSCRAWL_PROXY_PASSWORD") != "" {
        log.Fatal("set ADSCRAWL_PROXY_SERVER with proxy credentials")
    }

    routing.Viewport = &adscrawl.Viewport{Width: 1440, Height: 900}
    routing.UserAgentMode = "random"
    routing.UserAgentOS = "windows"
    routing.Fingerprint = &adscrawl.Fingerprint{
        WebRTC: "forward", WebGL: "random", WebGPU: "random", WebGLImage: "random",
        Canvas: "random", AudioContext: "random", ClientRects: "random",
        SpeechVoices: "random", Fonts: "random", Hardware: "random", DoNotTrack: "random",
    }

    png, err := client.Screenshot(context.Background(), adscrawl.ScreenshotOptions{
        PageOptions: adscrawl.PageOptions{
            BrowserSettings: routing,
            URL:             "https://www.browserscan.net/",
            WaitUntil:       "networkidle",
            TimeoutMS:       60000,
        },
        FullPage: true,
    })
    if err != nil {
        log.Fatal(err)
    }
    if err := os.WriteFile("browserscan.png", png, 0600); err != nil {
        log.Fatal(err)
    }
}
```

AdsCrawl uses real browsers with configurable routing and browser fingerprints. Randomized settings are generated as a coherent profile across the operating system, GPU, hardware, fonts, and related signals. This browser workflow has been verified to access and render BrowserScan, Pixelscan, and IPhey and return screenshots.

## Structured extraction

```go theme={null}
package main

import (
    "context"
    "fmt"
    "log"

    adscrawl "github.com/AdsCrawl/adscrawl-go"
)

func main() {
    client, err := adscrawl.NewClient(adscrawl.Config{})
    if err != nil {
        log.Fatal(err)
    }

    result, err := client.SPA.Extract(context.Background(), adscrawl.SPAOptions{
        URL: "https://www.adscrawl.net",
        Fields: map[string]adscrawl.Field{
            "title": {Source: "dom", Selector: "h1", Value: "text", Required: true},
        },
    })
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(result.Data["title"], result.MissingFields)
}
```

`client.SPA.Templates(ctx)` lists built-in templates. `client.SPA.Inspect(ctx, options)` suggests fields and schema.

## Remote CDP browsers

```go theme={null}
package main

import (
    "context"
    "fmt"
    "log"

    adscrawl "github.com/AdsCrawl/adscrawl-go"
)

func main() {
    client, err := adscrawl.NewClient(adscrawl.Config{})
    if err != nil {
        log.Fatal(err)
    }

    ctx := context.Background()
    session, err := client.CDP.Create(ctx, adscrawl.CreateSessionOptions{
        IdleTimeoutMS: 600000,
        MaxSessionMS:  3600000,
    })
    if err != nil {
        log.Fatal(err)
    }
    defer func() {
        if _, err := client.CDP.Close(ctx, session.SessionID); err != nil {
            log.Printf("close session: %v", err)
        }
    }()

    version, err := client.CDP.GetVersion(ctx, session)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(version.Browser)
    // Pass session.CDPBaseURL to a CDP-compatible browser automation library.
    // It contains a secret and must not be logged.
}
```

`GetVersion` validates the token URL and does not forward the API key.

## Persistent cloud browsers

```go theme={null}
package main

import (
    "context"
    "fmt"
    "log"
    "os"

    adscrawl "github.com/AdsCrawl/adscrawl-go"
)

func main() {
    server := os.Getenv("ADSCRAWL_PROXY_SERVER")
    if server == "" {
        log.Fatal("set ADSCRAWL_PROXY_SERVER before launching a cloud browser")
    }

    client, err := adscrawl.NewClient(adscrawl.Config{})
    if err != nil {
        log.Fatal(err)
    }

    ctx := context.Background()
    launched, err := client.CloudBrowsers.Launch(ctx, adscrawl.LaunchCloudBrowserOptions{
        StartCloudBrowserOptions: adscrawl.StartCloudBrowserOptions{
            Proxy: &adscrawl.Proxy{
                Server:   server,
                Username: os.Getenv("ADSCRAWL_PROXY_USERNAME"),
                Password: os.Getenv("ADSCRAWL_PROXY_PASSWORD"),
            },
        },
        Tabs: []any{"https://www.adscrawl.net"},
    })
    if err != nil {
        log.Fatal(err)
    }
    defer func() {
        if _, err := client.CloudBrowsers.Stop(ctx, launched.ID); err != nil {
            log.Printf("stop browser: %v", err)
        }
    }()

    profile, err := client.CloudBrowsers.Get(ctx, launched.ID)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(profile.ID, profile.Runtime.Status)
}
```

Cloud browser API-key starts and launches require an explicit top-level proxy. A `stopping` response means shutdown is pending; poll `Get` until `stopped`. A failed launch may return a cleanup profile ID in `*adscrawl.APIError`.

## Configuration and errors

Pass `Config{Timeout: 75 * time.Second}` to change the HTTP deadline. Set `PageOptions.TimeoutMS` to control the remote browser task separately.

```go theme={null}
var apiErr *adscrawl.APIError
if errors.As(err, &apiErr) {
    // apiErr.Status, apiErr.Code, apiErr.TraceID, apiErr.ID
}
```

`ADSCRAWL_API_KEY` supplies the key by default. The base URL defaults to `ADSCRAWL_BASE_URL`, then `ADSCRAWL_API_URL`, then `https://api.adscrawl.net`. Errors include `APIError`, `TimeoutError`, `ConnectionError`, and `ResponseError`. The HTTP deadline includes response body reading. Cancelling a request does not prove remote browser work stopped.


# JavaScript and TypeScript SDK for AdsCrawl
Source: https://docs.adscrawl.net/sdks/javascript

Install and use the AdsCrawl JavaScript SDK to fetch rendered content, screenshots, structured data, and remote CDP browser sessions. Node.js 20+, zero runtime dependencies.

The AdsCrawl JavaScript SDK works in Node.js 20 and later, supports both ESM and CommonJS, and ships with full TypeScript types and editor autocomplete. It uses native `fetch` and has zero runtime dependencies.

> [!NOTE]
>
> Keep your API key in server-side code. The SDK is not designed for use in browser bundles.

## Install

```bash theme={null}
npm install adscrawl
```

When you need to connect a remote browser with Playwright or Puppeteer, install the automation library separately:

```bash theme={null}
npm install adscrawl playwright-core
# or
npm install adscrawl puppeteer-core
```

> [card] [adscrawl-js on GitHub](https://github.com/AdsCrawl/adscrawl-js)
>
> View source, runnable examples, and releases.

## Authenticate and make your first request

[Create an API key](https://app.adscrawl.net/register/?utm_source=npm\&utm_medium=sdk\&utm_campaign=adscrawl-js) in the dashboard and set `ADSCRAWL_API_KEY` in your server environment. You can also pass the key explicitly to `new AdsCrawl({ apiKey: '...' })`.

```ts theme={null}
import AdsCrawl from 'adscrawl';

const client = new AdsCrawl();
const markdown = await client.markdown({
  url: 'https://www.adscrawl.net',
  waitUntil: 'domcontentloaded',
});
console.log(markdown);
```

CommonJS:

```js theme={null}
const { AdsCrawl } = require('adscrawl');

const client = new AdsCrawl();

async function main() {
  const markdown = await client.markdown({
    url: 'https://www.adscrawl.net',
    waitUntil: 'domcontentloaded',
  });
  console.log(markdown);
}

main().catch((error) => {
  console.error(error);
  process.exitCode = 1;
});
```

## Rendered content and screenshots

```ts theme={null}
import { writeFile } from 'node:fs/promises';
import AdsCrawl from 'adscrawl';

const client = new AdsCrawl();
const html = await client.html({ url: 'https://www.adscrawl.net' });
console.log(html);

const article = await client.article({ url: 'https://www.adscrawl.net' });
console.log(article.title, article.textContent);

const png = await client.screenshot({
  url: 'https://www.adscrawl.net',
  viewport: { width: 1440, height: 900 },
  fullPage: true,
  waitUntil: 'load',
});
await writeFile('page.png', png);
```

`html()` returns a string for `contentMode: 'html'` (the default) or `'markdown'`, and an `Article` for `'json'`. `markdown()` and `article()` are conveniences for those modes. `screenshot()` returns a `Uint8Array` containing a PNG.

## Proxy and fingerprint

Route through a custom proxy or use managed `countryCode` routing. Both cannot be combined.

```ts theme={null}
import AdsCrawl from 'adscrawl';

const client = new AdsCrawl();
const markdown = await client.markdown({
  url: 'https://www.adscrawl.net',
  countryCode: 'US',
  userAgentMode: 'random',
  userAgentOs: 'windows',
});
console.log(markdown);
```

AdsCrawl uses real browsers with configurable routing and browser fingerprints. Randomized settings are generated as a coherent profile across the operating system, GPU, hardware, fonts, and related signals. This browser workflow has been verified to access and render BrowserScan, Pixelscan, and IPhey and return screenshots.

This complete example opens BrowserScan and saves the returned PNG as `browserscan.png`:

```ts theme={null}
import { writeFile } from 'node:fs/promises';
import AdsCrawl from 'adscrawl';

const client = new AdsCrawl();
const server = process.env.ADSCRAWL_PROXY_SERVER;
const username = process.env.ADSCRAWL_PROXY_USERNAME;
const password = process.env.ADSCRAWL_PROXY_PASSWORD;
if (Boolean(username) !== Boolean(password)) {
  throw new Error('Set both proxy username and password, or neither.');
}
if (!server && (username || password)) {
  throw new Error('Set ADSCRAWL_PROXY_SERVER when supplying proxy credentials.');
}
const routing = server
  ? { proxy: username && password ? { server, username, password } : { server } }
  : { countryCode: 'GLOBAL' };

const png = await client.screenshot({
  url: 'https://www.browserscan.net/',
  ...routing,
  viewport: { width: 1440, height: 900 },
  fullPage: true,
  waitUntil: 'networkidle',
  timeoutMs: 60_000,
  userAgentMode: 'random',
  userAgentOs: 'windows',
  fingerprint: {
    webRtc: 'forward',
    webGl: 'random',
    webGpu: 'random',
    webGlImage: 'random',
    canvas: 'random',
    audioContext: 'random',
    clientRects: 'random',
    speechVoices: 'random',
    fonts: 'random',
    hardware: 'random',
    doNotTrack: 'random',
  },
}, { timeoutMs: 75_000 });

const output = 'browserscan.png';
await writeFile(output, png);
console.log(`Saved fingerprint-check screenshot to ${output}`);
```

## Structured extraction

List available templates and their parameters:

```ts theme={null}
const { templates } = await client.spa.templates();
console.log(templates);

const result = await client.spa.extract({
  template: 'google-trends-explore',
  keyword: 'playwright,puppeteer',
});
console.log(result.data);
```

For your own page, specify DOM or network fields. A generic describes the expected output but does not validate your custom data at runtime. Always check `missingFields`.

```ts theme={null}
const result = await client.spa.extract<{ title: string }>({
  url: 'https://www.adscrawl.net',
  fields: {
    title: { source: 'dom', selector: 'h1', value: 'text', required: true },
  },
});
console.log(result.data.title);
console.log(result.missingFields);

const inspection = await client.spa.inspect({ url: 'https://www.adscrawl.net' });
console.log(inspection.candidates);
```

Use `actions` for clicks, input, scrolling, and waits, and `waitFor` for a visible selector or text. See the [API reference](/api-reference/spa-extract) for template-specific requirements.

## Remote CDP browsers

Connect a Playwright browser over CDP to an AdsCrawl remote session:

```ts theme={null}
import { chromium } from 'playwright-core';
import AdsCrawl from 'adscrawl';

const client = new AdsCrawl();
const session = await client.cdp.create({
  idleTimeoutMs: 600_000,
  maxSessionMs: 3_600_000,
  browserSettings: { viewport: { width: 1440, height: 900 } },
});

try {
  const browser = await chromium.connectOverCDP(session.cdpBaseUrl);
  const context = browser.contexts()[0] ?? await browser.newContext();
  const page = context.pages()[0] ?? await context.newPage();
  await page.goto('https://www.adscrawl.net');
  console.log(await page.title());
} finally {
  await client.cdp.close(session.sessionId);
}
```

The creation response contains `sessionId`, `expiresAt`, and `cdpBaseUrl`. It does not contain a WebSocket URL. For Puppeteer, use discovery:

```ts theme={null}
import puppeteer from 'puppeteer-core';
import AdsCrawl from 'adscrawl';

const client = new AdsCrawl();
const session = await client.cdp.create();
try {
  const version = await client.cdp.getVersion(session);
  const browser = await puppeteer.connect({
    browserWSEndpoint: version.webSocketDebuggerUrl,
  });
  try {
    const pages = await browser.pages();
    const page = pages[0] ?? await browser.newPage();
    await page.goto('https://www.adscrawl.net');
    console.log(await page.title());
  } finally {
    browser.disconnect();
  }
} finally {
  await client.cdp.close(session.sessionId);
}
```

`cdp.list()` returns `{ ok, data }`. `cdp.liveToken(sessionId)` returns a single-use live-control URL valid for 30 seconds. Treat all connection URLs as secrets and do not log them.

## Persistent cloud browsers

Cloud browser profiles retain their configuration after stop. API-key starts require an explicit, top-level custom proxy on every start, even when a proxy was saved in the profile.

Set `ADSCRAWL_PROXY_SERVER` to your HTTP or SOCKS5 proxy URL with an explicit port. If authentication is needed, also set `ADSCRAWL_PROXY_USERNAME` and `ADSCRAWL_PROXY_PASSWORD`.

```ts theme={null}
import { setTimeout as delay } from 'node:timers/promises';
import AdsCrawl, {
  AdsCrawlAPIError,
  AdsCrawlConnectionError,
  AdsCrawlTimeoutError,
} from 'adscrawl';

const client = new AdsCrawl();
const server = process.env.ADSCRAWL_PROXY_SERVER;
if (!server) throw new Error('Set ADSCRAWL_PROXY_SERVER to your proxy URL.');
const username = process.env.ADSCRAWL_PROXY_USERNAME;
const password = process.env.ADSCRAWL_PROXY_PASSWORD;
if (Boolean(username) !== Boolean(password)) {
  throw new Error('Set both proxy username and password, or neither.');
}
const proxy = username && password ? { server, username, password } : { server };

async function stopAndWait(id: string) {
  const end = Date.now() + 120_000;
  while (Date.now() < end) {
    try {
      const stopped = await client.cloudBrowsers.stop(id, {
        timeoutMs: Math.max(1, Math.min(10_000, end - Date.now())),
      });
      if (stopped.runtime.status === 'stopped') return;
    } catch (error) {
      const retryable = error instanceof AdsCrawlTimeoutError
        || error instanceof AdsCrawlConnectionError
        || (error instanceof AdsCrawlAPIError
          && (error.code === 'CDP_SESSION_STARTING' || error.status >= 500));
      if (!retryable) throw error;
    }
    const remaining = end - Date.now();
    if (remaining > 0) await delay(Math.min(2000, remaining));
  }
  throw new Error(`Stop unconfirmed for ${id}; inspect the profile and retry stop.`);
}

const { id } = await client.cloudBrowsers.create({ remark: 'My workflow' });
try {
  await client.cloudBrowsers.start(id, { proxy });
  const profile = await client.cloudBrowsers.get(id);
  console.log({ id: profile.id, status: profile.runtime.status });
  // profile.runtime.connectUrl opens the interactive browser in an
  // authenticated browser signed in as the profile owner.
} finally {
  await stopAndWait(id);
}
```

`cloudBrowsers.launch({ proxy, tabs?, cookies?, fingerprint? })` creates a persistent profile and starts it in one request. Failed launches can return a profile id in `AdsCrawlAPIError.id`; inspect and stop that profile. If no id was received, inspect `cloudBrowsers.list()` before repeating the launch.

`cloudBrowsers.list({ page?, pageSize? })` includes pagination and saved/running quotas. A stop response with `runtime.status: 'stopping'` means shutdown is pending and quota is still reserved. Closing the viewer does not stop billing. The SDK covers API-key endpoints; profile configuration `PATCH`/`DELETE` and viewer authentication require a dashboard session.

## Configuration, deadlines, and cancellation

```ts theme={null}
import AdsCrawl from 'adscrawl';

const client = new AdsCrawl({
  apiKey: process.env.ADSCRAWL_API_KEY,
  baseURL: 'https://api.adscrawl.net',
  timeoutMs: 90_000,
  // fetch: customFetch,
});

const controller = new AbortController();
const result = await client.markdown(
  { url: 'https://www.adscrawl.net', timeoutMs: 60_000 }, // Server task timeout.
  { timeoutMs: 75_000, signal: controller.signal }, // HTTP deadline / cancellation.
);
console.log(result);
```

The API key defaults to `ADSCRAWL_API_KEY`. The API origin defaults to `ADSCRAWL_BASE_URL`, then `ADSCRAWL_API_URL`, then `https://api.adscrawl.net`. HTTP deadlines cover both the request and reading its response. Ordinary calls default to 90 seconds; `cloudBrowsers.launch()` defaults to 195 seconds to allow startup and server cleanup. For long tasks, set an HTTP deadline greater than the server task timeout.

> [!WARNING]
>
> Cancellation or a deadline does not prove remote work stopped.

## Errors

```ts theme={null}
import AdsCrawl, { AdsCrawlAPIError, AdsCrawlTimeoutError } from 'adscrawl';

const client = new AdsCrawl();

try {
  const markdown = await client.markdown({ url: 'https://www.adscrawl.net' });
  console.log(markdown);
} catch (error) {
  if (error instanceof AdsCrawlAPIError) {
    console.error(error.status, error.code, error.traceId);
    // error.body contains the service response with credentials redacted.
  } else if (error instanceof AdsCrawlTimeoutError) {
    console.error('Request timed out. Inspect sessions/profiles before retrying browser creation.');
  } else {
    throw error;
  }
}
```

API errors preserve HTTP `status`, stable `code`, redacted `body`, `traceId`, and `requestId` when available. Network failures use `AdsCrawlConnectionError`; invalid or empty responses use `AdsCrawlResponseError`. Caller cancellation preserves the `AbortSignal` reason. Requests are never automatically retried.


# AdsCrawl SDKs and Integrations
Source: https://docs.adscrawl.net/sdks/overview

Official AdsCrawl SDKs for JavaScript, Python, Go, Ruby, and PHP. Zero runtime dependencies, server-side use, and no automatic retries of metered requests.

Official AdsCrawl SDKs turn a URL into rendered HTML, Markdown, structured data, or a PNG screenshot. When your workflow needs interaction, connect a remote browser with Playwright or Puppeteer. All SDKs share the same design: zero runtime dependencies, explicit HTTP deadlines, typed errors, and no automatic retries of metered requests or browser creationOfficial AdsCrawl SDKs turn a URL into rendered HTML, Markdown, structured data, or a PNG screenshot. When your workflow needs interaction, connect a remote browser with Playwright or Puppeteer. All SDKs share the same design: zero runtime dependencies, explicit HTTP deadlines, typed errors, and no automatic retries of metered requests or browser creation.

## Supported languageupported languages

| Language | Install command | Version requirement | Repository |
| - | - | - | - |
| JavaScript / TypeScript | `npm install adscrawl` | Node.js 20+ | [adscrawl-js](https://github.com/AdsCrawl/adscrawl-js) |
| Python | `pip install adscrawl` | Python 3.9+ | [adscrawl-python](https://github.com/AdsCrawl/adscrawl-python) |
| Go | `go get github.com/AdsCrawl/adscrawl-go@v0.1.0` | Go 1.21+ | [adscrawl-go](https://github.com/AdsCrawl/adscrawl-go) |
| Ruby | `gem install adscrawl` | Ruby 3.1+ | [adscrawl-ruby](https://github.com/AdsCrawl/adscrawl-ruby) |
| PHP | `composer require adscrawl/adscrawl` | PHP 8.1+ | [adscrawl-php](https://github.com/AdsCrawl/adscrawl-php) |

| Language | Install command | Version requirement | Repository |
| - | - | - | - |
| JavaScript / TypeScript | `npm install adscrawl` | Node.js 20+ | [adscrawl-js](https://github.com/AdsCrawl/adscrawl-js) |
| Python | `pip install adscrawl` | Python 3.9+ | [adscrawl-python](https://github.com/AdsCrawl/adscrawl-python) |
| Go | `go get github.com/AdsCrawl/adscrawl-go@v0.1.0` | Go 1.21+ | [adscrawl-go](https://github.com/AdsCrawl/adscrawl-go) |
| Ruby | `gem install adscrawl` | Ruby 3.1+ | [adscrawl-ruby](https://github.com/AdsCrawl/adscrawl-ruby) |
| PHP | `composer require adscrawl/adscrawl` | PHP 8.1+ | [adscrawl-php](https://github.com/AdsCrawl/adscrawl-php) |

## Corere capabilities

* **Rendered content and screenshots** — fetch HTML, Markdown, or an article object from any URL, or capture a PNG screenshot with viewport, full-page, and element-level targeting.
* **Structured extraction and templates** — extract named fields from pages using DOM selectors or network responses, or use built-in SPA templates like `google-trends-explore`.
* **Remote CDP sessions** — create short-lived browser sessions and connect your own Playwright, Puppeteer, or CDP-compatible library to `cdpBaseUrl`.
* **Persistent cloud browsers** — create long-lived browser profiles, start and stop them on demand, and retain cookies, tabs, and configuration across sessions.
* **Proxy and fingerprinting** — route through custom HTTP or SOCKS5 proxies, or use managed `countryCode` routing. Apply coherent browser fingerprints verified against BrowserScan, Pixelscan, and IPhey.

## Common configuration

All SDKs resolve configuration in the same order:

* **API key:** defaults to the `ADSCRAWL_API_KEY` environment variable. You can also pass it explicitly during client initialization.
* **Base URL:** defaults to `ADSCRAWL_BASE_URL`, then `ADSCRAWL_API_URL`, then `https://api.adscrawl.net`.
* **HTTP timeout:** defaults to 90 seconds for most calls. `cloudBrowsers.launch()` uses a longer default to allow startup and server cleanup.

All SDKs are server-side only. Do not expose your API key in frontend bundles.

## Quick examples

The snippet below fetches a Markdown version of the AdsCrawl homepage in every supported language.

* **Rendered content and screenshots** — fetch HTML, Markdown, or an article object from any URL, or capture a PNG screenshot with viewport, full-page, and element-level targeting.
* **Structured extraction and templates** — extract named fields from pages using DOM selectors or network responses, or use built-in SPA templates like `google-trends-explore`.
* **Remote CDP sessions** — create short-lived browser sessions and connect your own Playwright, Puppeteer, or CDP-compatible library to `cdpBaseUrl`.
* **Persistent cloud browsers** — create long-lived browser profiles, start and stop them on demand, and retain cookies, tabs, and configuration across sessions.
* **Proxy and fingerprinting** — route through custom HTTP or SOCKS5 proxies, or use managed `countryCode` routing. Apply coherent browser fingerprints verified against BrowserScan, Pixelscan, and IPhey.

## Common configuration

All SDKs resolve configuration in the same order:

* **API key:** defaults to the `ADSCRAWL_API_KEY` environment variable. You can also pass it explicitly during client initialization.
* **Base URL:** defaults to `ADSCRAWL_BASE_URL`, then `ADSCRAWL_API_URL`, then `https://api.adscrawl.net`.
* **HTTP timeout:** defaults to 90 seconds for most calls. `cloudBrowsers.launch()` uses a longer default to allow startup and server cleanup.

All SDKs are server-side only. Do not expose your API key in frontend bundles.

## Quick examples

The snippet below fetches a Markdown version of the AdsCrawl homepage in every supported language.

```ts JavaScript / TypeScript theme={null}
import AdsCrawl from 'adscrawl';

const client = new AdsCrawl();
const markdown = await client.markdown({
  url: 'https://www.adscrawl.net',
  waitUntil: 'domcontentloaded',
});
console.log(markdown);import AdsCrawl from 'adscrawl';

const client = new AdsCrawl();
const markdown = await client.markdown({
  url: 'https://www.adscrawl.net',
  waitUntil: 'domcontentloaded',
});
console.log(markdown);
```

```python Python theme={null}
from adscrawl import AdsCrawl

client = AdsCrawl()
markdown = client.markdown({
    "url": "https://www.adscrawl.net",
    "waitUntil": "domcontentloaded",
})
print(markdown)from adscrawl import AdsCrawl

client = AdsCrawl()
markdown = client.markdown({
    "url": "https://www.adscrawl.net",
    "waitUntil": "domcontentloaded",
})
print(markdown)
```

```go Go theme={null}
package main

import (
    "context"
    "fmt"
    "log"

    adscrawl "github.com/AdsCrawl/adscrawl-go"
)

func main() {
    client, err := adscrawl.NewClient(adscrawl.Config{})
    if err != nil { log.Fatal(err) }
    markdown, err := client.Markdown(context.Background(), adscrawl.ContentOptions{
        PageOptions: adscrawl.PageOptions{
            URL:       "https://www.adscrawl.net",
            WaitUntil: "domcontentloaded",
        },
    })
    if err != nil { log.Fatal(err) }
    fmt.Println(markdown)
}package main

import (
    "context"
    "fmt"
    "log"

    adscrawl "github.com/AdsCrawl/adscrawl-go"
)

func main() {
    client, err := adscrawl.NewClient(adscrawl.Config{})
    if err != nil { log.Fatal(err) }
    markdown, err := client.Markdown(context.Background(), adscrawl.ContentOptions{
        PageOptions: adscrawl.PageOptions{
            URL:       "https://www.adscrawl.net",
            WaitUntil: "domcontentloaded",
        },
    })
    if err != nil { log.Fatal(err) }
    fmt.Println(markdown)
}
```

```ruby Ruby theme={null}
require "adscrawl"

client = AdsCrawl::Client.new
markdown = client.markdown(
  url: "https://www.adscrawl.net",
  waitUntil: "domcontentloaded"
)
puts markdownrequire "adscrawl"

client = AdsCrawl::Client.new
markdown = client.markdown(
  url: "https://www.adscrawl.net",
  waitUntil: "domcontentloaded"
)
puts markdown
```

```php PHP theme={null}
<?php

require __DIR__ . '/vendor/autoload.php';

use AdsCrawl\Client;

$client = new Client();
$markdown = $client->markdown([
    'url' => 'https://www.adscrawl.net',
    'waitUntil' => 'domcontentloaded',
]);
echo $markdown;<?php

require __DIR__ . '/vendor/autoload.php';

use AdsCrawl\Client;

$client = new Client();
$markdown = $client->markdown([
    'url' => 'https://www.adscrawl.net',
    'waitUntil' => 'domcontentloaded',
]);
echo $markdown;
```

## Next steps

> [card] [JavaScript SDK](/sdks/javascript)
>
> TypeScript types, ESM/CommonJS support, and native `fetch`. Node.js 20+.

> [card] [Python SDK](/sdks/python)
>
> Sync and async clients, zero runtime dependencies. Python 3.9+.

> [card] [Go SDK](/sdks/go)
>
> Context-aware deadlines, typed models, standard-library HTTP. Go 1.21+.

> [card] [Ruby SDK](/sdks/ruby)
>
> Keyword and hash arguments, standard-library only. Ruby 3.1+.

> [card] [PHP SDK](/sdks/php)
>
> Requires JSON and cURL extensions only. PHP 8.1+.

> [card] [Codex Plugin](/sdks/codex)
>
> OpenAI Codex MCP integration for browser automation from natural language.

## Next steps

> [card] [JavaScript SDK](/sdks/javascript)
>
> TypeScript types, ESM/CommonJS support, and native `fetch`. Node.js 20+.

> [card] [Python SDK](/sdks/python)
>
> Sync and async clients, zero runtime dependencies. Python 3.9+.

> [card] [Go SDK](/sdks/go)
>
> Context-aware deadlines, typed models, standard-library HTTP. Go 1.21+.

> [card] [Ruby SDK](/sdks/ruby)
>
> Keyword and hash arguments, standard-library only. Ruby 3.1+.

> [card] [PHP SDK](/sdks/php)
>
> Requires JSON and cURL extensions only. PHP 8.1+.

> [card] [Codex Plugin](/sdks/codex)
>
> OpenAI Codex MCP integration for browser automation from natural language.


# PHP SDK for AdsCrawl
Source: https://docs.adscrawl.net/sdks/php

Install and use the AdsCrawl PHP SDK to fetch rendered content, screenshots, structured data, and manage remote CDP sessions. PHP 8.1+, standard JSON and cURL extensions only.

The AdsCrawl PHP SDK works with PHP 8.1 and later. It requires only the standard JSON and cURL extensions, uses explicit HTTP deadlines, and returns credential-redacted errors.

> [!NOTE]
>
> Keep your API key in server-side code. Never expose it in client-side applications.

## Install

```bash theme={null}
composer require adscrawl/adscrawl
```

> [card] [adscrawl-php on GitHub](https://github.com/AdsCrawl/adscrawl-php)
>
> View source, runnable examples, and releases.

## Authenticate and make your first request

[Create an API key](https://app.adscrawl.net/register/?utm_source=packagist\&utm_medium=sdk\&utm_campaign=adscrawl-php) in the dashboard and set `ADSCRAWL_API_KEY` in your server environment. You can also pass the key to `new Client(apiKey: '...')`.

```php theme={null}
<?php

require __DIR__ . '/vendor/autoload.php';

use AdsCrawl\Client;

$client = new Client();
$markdown = $client->markdown([
    'url' => 'https://www.adscrawl.net',
    'waitUntil' => 'domcontentloaded',
]);
echo $markdown;
```

## Rendered content and screenshots

```php theme={null}
<?php

require __DIR__ . '/vendor/autoload.php';

use AdsCrawl\Client;

$client = new Client();
$html = $client->html(['url' => 'https://www.adscrawl.net']);
$article = $client->article(['url' => 'https://www.adscrawl.net']);
echo $article['title'] . "\n" . $article['textContent'];

$png = $client->screenshot([
    'url' => 'https://www.adscrawl.net',
    'viewport' => ['width' => 1440, 'height' => 900],
    'fullPage' => true,
    'waitUntil' => 'load',
]);
file_put_contents('page.png', $png);
```

`html()` returns HTML, `markdown()` returns Markdown, `article()` returns an associative array, and `screenshot()` returns PNG bytes. Page options include `viewport`, `locale`, `cookies`, custom `proxy`, managed `countryCode`, `fingerprint`, server-side `timeoutMs`, and navigation `waitUntil`. A custom proxy and managed country cannot be combined.

## Proxy and fingerprint

```php theme={null}
<?php

require __DIR__ . '/vendor/autoload.php';

use AdsCrawl\Client;

$client = new Client();
$server = getenv('ADSCRAWL_PROXY_SERVER') ?: null;
$username = getenv('ADSCRAWL_PROXY_USERNAME') ?: null;
$password = getenv('ADSCRAWL_PROXY_PASSWORD') ?: null;
if (($username === null) !== ($password === null)) {
    throw new RuntimeException('Set both proxy username and password, or neither.');
}
if ($server === null && ($username !== null || $password !== null)) {
    throw new RuntimeException('Set ADSCRAWL_PROXY_SERVER with proxy credentials.');
}

$routing = ['countryCode' => 'GLOBAL'];
if ($server !== null) {
    $proxy = ['server' => $server];
    if ($username !== null && $password !== null) {
        $proxy += ['username' => $username, 'password' => $password];
    }
    $routing = ['proxy' => $proxy];
}

$png = $client->screenshot([
    'url' => 'https://www.browserscan.net/',
    ...$routing,
    'viewport' => ['width' => 1440, 'height' => 900],
    'fullPage' => true,
    'waitUntil' => 'networkidle',
    'timeoutMs' => 60_000,
    'userAgentMode' => 'random',
    'userAgentOs' => 'windows',
    'fingerprint' => [
        'webRtc' => 'forward', 'webGl' => 'random', 'webGpu' => 'random',
        'webGlImage' => 'random', 'canvas' => 'random',
        'audioContext' => 'random', 'clientRects' => 'random',
        'speechVoices' => 'random', 'fonts' => 'random',
        'hardware' => 'random', 'doNotTrack' => 'random',
    ],
], timeoutMs: 75_000);
file_put_contents('browserscan.png', $png);
```

AdsCrawl uses real browsers with configurable routing and browser fingerprints. Randomized settings are generated as a coherent profile across the operating system, GPU, hardware, fonts, and related signals. This browser workflow has been verified to access and render BrowserScan, Pixelscan, and IPhey and return screenshots.

## Structured extraction

```php theme={null}
<?php

require __DIR__ . '/vendor/autoload.php';

use AdsCrawl\Client;

$client = new Client();
print_r($client->spa->templates()['templates']);

$result = $client->spa->extract([
    'url' => 'https://www.adscrawl.net',
    'fields' => [
        'title' => ['source' => 'dom', 'selector' => 'h1', 'value' => 'text', 'required' => true],
    ],
]);
echo $result['data']['title'];

$inspection = $client->spa->inspect(['url' => 'https://www.adscrawl.net']);
print_r($inspection['candidates']);
```

## Remote CDP browsers

```php theme={null}
<?php

require __DIR__ . '/vendor/autoload.php';

use AdsCrawl\Client;

$client = new Client();
$session = $client->cdp->create([
    'idleTimeoutMs' => 600_000,
    'maxSessionMs' => 3_600_000,
    'browserSettings' => ['viewport' => ['width' => 1440, 'height' => 900]],
]);
try {
    $version = $client->cdp->getVersion($session);
    echo $version['Browser'];
    // Connect a CDP-compatible PHP library to $session['cdpBaseUrl'].
} finally {
    $client->cdp->close($session['sessionId']);
}
```

`$client->cdp->list()` returns active sessions. `getVersion()` validates the session token URL and deliberately omits the API key. CDP connection URLs contain secrets; do not log them.

## Persistent cloud browsers

```php theme={null}
<?php

require __DIR__ . '/vendor/autoload.php';

use AdsCrawl\Client;

$client = new Client();
$launched = $client->cloudBrowsers->launch([
    'proxy' => ['server' => getenv('ADSCRAWL_PROXY_SERVER')],
    'tabs' => ['https://www.adscrawl.net'],
]);
$browserId = $launched['id'];
try {
    $profile = $client->cloudBrowsers->get($browserId);
    echo $profile['runtime']['status'];
} finally {
    $client->cloudBrowsers->stop($browserId);
}
```

API-key starts and launches require an explicit top-level custom proxy. A `stopping` response means shutdown is pending; poll `get()` until `stopped`. A failed launch may expose a cleanup profile ID as `ApiException::$resourceId`.

## Configuration and errors

```php theme={null}
<?php

require __DIR__ . '/vendor/autoload.php';

use AdsCrawl\Client;
use AdsCrawl\Exception\ApiException;
use AdsCrawl\Exception\TimeoutException;

$client = new Client(baseUrl: 'https://api.adscrawl.net', timeoutMs: 90_000);
try {
    echo $client->markdown(
        ['url' => 'https://www.adscrawl.net', 'timeoutMs' => 60_000],
        timeoutMs: 75_000,
    );
} catch (ApiException $error) {
    echo $error->status . ' ' . $error->apiCode . ' ' . $error->traceId;
} catch (TimeoutException) {
    echo 'Inspect remote sessions before retrying.';
}
```

The API key defaults to `ADSCRAWL_API_KEY`. The base URL defaults to `ADSCRAWL_BASE_URL`, then `ADSCRAWL_API_URL`, then `https://api.adscrawl.net`. Errors include `ApiException`, `TimeoutException`, `ConnectionException`, and `ResponseException`. API response bodies are credential-redacted. The HTTP deadline includes response body reading. Requests are not automatically retried.


# Python SDK for AdsCrawl
Source: https://docs.adscrawl.net/sdks/python

Install and use the AdsCrawl Python SDK to fetch rendered content, screenshots, structured data, and remote CDP browser sessions. Python 3.9+, zero runtime dependencies.

The AdsCrawl Python SDK works with Python 3.9 and later. It provides both synchronous and asynchronous clients, uses only the Python standard library for HTTP, and has zero runtime dependencies.

> [!NOTE]
>
> Keep your API key in server-side code. Never expose it in client-side applications.

## Install

```bash theme={null}
pip install adscrawl
```

When you need to connect a remote browser with Playwright, install the extra:

```bash theme={null}
pip install "adscrawl[playwright]"
playwright install chromium
```

> [card] [adscrawl-python on GitHub](https://github.com/AdsCrawl/adscrawl-python)
>
> View source, runnable examples, and releases.

## Authenticate and make your first request

[Create an API key](https://app.adscrawl.net/register/?utm_source=pypi\&utm_medium=sdk\&utm_campaign=adscrawl-python) in the dashboard and set `ADSCRAWL_API_KEY` in your server environment. You can also pass the key explicitly to `AdsCrawl(api_key="...")`.

```python theme={null}
from adscrawl import AdsCrawl

client = AdsCrawl()
markdown = client.markdown({
    "url": "https://www.adscrawl.net",
    "waitUntil": "domcontentloaded",
})
print(markdown)
```

## Rendered content and screenshots

```python theme={null}
from pathlib import Path
from adscrawl import AdsCrawl

client = AdsCrawl()
html = client.html({"url": "https://www.adscrawl.net"})
article = client.article({"url": "https://www.adscrawl.net"})
print(article["title"], article["textContent"])

png = client.screenshot({
    "url": "https://www.adscrawl.net",
    "viewport": {"width": 1440, "height": 900},
    "fullPage": True,
    "waitUntil": "load",
})
Path("page.png").write_bytes(png)
```

`html()` returns HTML by default. `markdown()` returns Markdown, `article()` returns a dictionary, and `screenshot()` returns PNG bytes. Use a custom `proxy` or managed `countryCode`; they cannot be combined.

## Proxy and fingerprint

```python theme={null}
import os
from pathlib import Path
from adscrawl import AdsCrawl

client = AdsCrawl()
server = os.getenv("ADSCRAWL_PROXY_SERVER")
username = os.getenv("ADSCRAWL_PROXY_USERNAME")
password = os.getenv("ADSCRAWL_PROXY_PASSWORD")
if bool(username) != bool(password):
    raise RuntimeError("Set both proxy username and password, or neither.")
if not server and (username or password):
    raise RuntimeError("Set ADSCRAWL_PROXY_SERVER with proxy credentials.")

routing = {"countryCode": "GLOBAL"}
if server:
    proxy = {"server": server}
    if username and password:
        proxy.update({"username": username, "password": password})
    routing = {"proxy": proxy}

png = client.screenshot({
    "url": "https://www.browserscan.net/",
    **routing,
    "viewport": {"width": 1440, "height": 900},
    "fullPage": True,
    "waitUntil": "networkidle",
    "timeoutMs": 60_000,
    "userAgentMode": "random",
    "userAgentOs": "windows",
    "fingerprint": {
        "webRtc": "forward", "webGl": "random", "webGpu": "random",
        "webGlImage": "random", "canvas": "random",
        "audioContext": "random", "clientRects": "random",
        "speechVoices": "random", "fonts": "random",
        "hardware": "random", "doNotTrack": "random",
    },
}, timeout_ms=75_000)
Path("browserscan.png").write_bytes(png)
```

AdsCrawl uses real browsers with configurable routing and browser fingerprints. Randomized settings are generated as a coherent profile across the operating system, GPU, hardware, fonts, and related signals. This browser workflow has been verified to access and render BrowserScan, Pixelscan, and IPhey and return screenshots.

## Structured extraction

```python theme={null}
from adscrawl import AdsCrawl

client = AdsCrawl()
print(client.spa.templates()["templates"])

result = client.spa.extract({
    "url": "https://www.adscrawl.net",
    "fields": {
        "title": {"source": "dom", "selector": "h1", "value": "text", "required": True},
    },
})
print(result["data"]["title"], result["missingFields"])

inspection = client.spa.inspect({"url": "https://www.adscrawl.net"})
print(inspection["candidates"])
```

A generic type argument describes the expected output but does not validate your custom data at runtime. Always check `missingFields`.

## Remote CDP browsers

```python theme={null}
from adscrawl import AdsCrawl
from playwright.sync_api import sync_playwright

client = AdsCrawl()
session = client.cdp.create({
    "idleTimeoutMs": 600_000,
    "maxSessionMs": 3_600_000,
    "browserSettings": {"viewport": {"width": 1440, "height": 900}},
})
try:
    with sync_playwright() as playwright:
        browser = playwright.chromium.connect_over_cdp(session["cdpBaseUrl"])
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()
        page.goto("https://www.adscrawl.net")
        print(page.title())
        browser.close()
finally:
    client.cdp.close(session["sessionId"])
```

`cdp.list()` returns `{"ok": True, "data": [...]}`. `cdp.get_version(session)` discovers the WebSocket endpoint without forwarding the API key. Connection URLs contain secrets and should not be logged.

## Persistent cloud browsers

```python theme={null}
import os
from adscrawl import AdsCrawl

client = AdsCrawl()
server = os.environ["ADSCRAWL_PROXY_SERVER"]
proxy = {"server": server}
launched = client.cloud_browsers.launch({
    "proxy": proxy,
    "tabs": ["https://www.adscrawl.net"],
})
browser_id = launched["id"]
try:
    profile = client.cloud_browsers.get(browser_id)
    print(profile["id"], profile["runtime"]["status"])
finally:
    client.cloud_browsers.stop(browser_id)
```

API-key starts and launches require a top-level custom `proxy` every time. A `stopping` response does not confirm shutdown; poll `get()` until `stopped`. Failed launches can expose a cleanup id as `AdsCrawlAPIError.id`.

## Async client

```python theme={null}
import asyncio
from adscrawl import AsyncAdsCrawl

async def main():
    async with AsyncAdsCrawl() as client:
        markdown = await client.markdown({"url": "https://www.adscrawl.net"})
        print(markdown)

asyncio.run(main())
```

The async facade runs the dependency-free standard-library HTTP transport in worker threads. Cancelling a coroutine or reaching a timeout does not prove remote browser work stopped.

## Configuration and errors

```python theme={null}
from adscrawl import AdsCrawl, AdsCrawlAPIError, AdsCrawlTimeoutError

client = AdsCrawl(base_url="https://api.adscrawl.net", timeout_ms=90_000)
try:
    text = client.markdown(
        {"url": "https://www.adscrawl.net", "timeoutMs": 60_000},
        timeout_ms=75_000,
    )
    print(text)
except AdsCrawlAPIError as error:
    print(error.status, error.code, error.trace_id)
except AdsCrawlTimeoutError:
    print("Inspect remote sessions before retrying.")
```

The API key defaults to `ADSCRAWL_API_KEY`. The base URL defaults to `ADSCRAWL_BASE_URL`, then `ADSCRAWL_API_URL`, then `https://api.adscrawl.net`. Errors include `AdsCrawlAPIError`, `AdsCrawlTimeoutError`, `AdsCrawlConnectionError`, and `AdsCrawlResponseError`. API response bodies are credential-redacted. The HTTP deadline includes response body reading. Requests are not automatically retried.


# Ruby SDK for AdsCrawl
Source: https://docs.adscrawl.net/sdks/ruby

Install and use the AdsCrawl Ruby SDK to fetch rendered content, screenshots, structured data, and manage remote CDP sessions. Ruby 3.1+, standard library only.

The AdsCrawl Ruby SDK works with Ruby 3.1 and later. It uses only the Ruby standard library, supports hash and keyword arguments that mirror the HTTP API's camelCase field names, and provides explicit HTTP deadlines with credential-redacted errors.

> [!NOTE]
>
> Keep your API key in server-side code. Never expose it in client-side applications.

## Install

```bash theme={null}
gem install adscrawl
```

Or add it to your bundle:

```bash theme={null}
bundle add adscrawl
```

> [card] [adscrawl-ruby on GitHub](https://github.com/AdsCrawl/adscrawl-ruby)
>
> View source, runnable examples, and releases.

## Authenticate and make your first request

[Create an API key](https://app.adscrawl.net/register/?utm_source=rubygems\&utm_medium=sdk\&utm_campaign=adscrawl-ruby) in the dashboard and set `ADSCRAWL_API_KEY` in your server environment. You can also pass the key to `AdsCrawl::Client.new(api_key: "...")`.

```ruby theme={null}
require "adscrawl"

client = AdsCrawl::Client.new
markdown = client.markdown(
  url: "https://www.adscrawl.net",
  waitUntil: "domcontentloaded"
)
puts markdown
```

## Rendered content and screenshots

```ruby theme={null}
require "adscrawl"

client = AdsCrawl::Client.new
html = client.html(url: "https://www.adscrawl.net")
article = client.article(url: "https://www.adscrawl.net")
puts [article["title"], article["textContent"]]

png = client.screenshot(
  url: "https://www.adscrawl.net",
  viewport: {width: 1440, height: 900},
  fullPage: true,
  waitUntil: "load"
)
File.binwrite("page.png", png)
```

`html` returns HTML, `markdown` returns Markdown, `article` returns a Hash, and `screenshot` returns binary PNG data. Page options include `viewport`, `locale`, `cookies`, custom `proxy`, managed `countryCode`, `fingerprint`, server-side `timeoutMs`, and navigation `waitUntil`. A custom proxy and managed country cannot be combined.

## Proxy and fingerprint

```ruby theme={null}
require "adscrawl"

client = AdsCrawl::Client.new
server = ENV["ADSCRAWL_PROXY_SERVER"]
username = ENV["ADSCRAWL_PROXY_USERNAME"]
password = ENV["ADSCRAWL_PROXY_PASSWORD"]
raise "Set both proxy username and password, or neither." if username.nil? != password.nil?
raise "Set ADSCRAWL_PROXY_SERVER with proxy credentials." if server.nil? && (username || password)

routing = {countryCode: "GLOBAL"}
if server
  proxy = {server: server}
  proxy.merge!(username: username, password: password) if username && password
  routing = {proxy: proxy}
end

png = client.screenshot(
  **routing,
  url: "https://www.browserscan.net/",
  viewport: {width: 1440, height: 900},
  fullPage: true,
  waitUntil: "networkidle",
  timeoutMs: 60_000,
  userAgentMode: "random",
  userAgentOs: "windows",
  fingerprint: {
    webRtc: "forward", webGl: "random", webGpu: "random",
    webGlImage: "random", canvas: "random", audioContext: "random",
    clientRects: "random", speechVoices: "random", fonts: "random",
    hardware: "random", doNotTrack: "random"
  },
  timeout_ms: 75_000
)
File.binwrite("browserscan.png", png)
```

AdsCrawl uses real browsers with configurable routing and browser fingerprints. Randomized settings are generated as a coherent profile across the operating system, GPU, hardware, fonts, and related signals. This browser workflow has been verified to access and render BrowserScan, Pixelscan, and IPhey and return screenshots.

## Structured extraction

```ruby theme={null}
require "adscrawl"

client = AdsCrawl::Client.new
puts client.spa.templates["templates"]

result = client.spa.extract(
  url: "https://www.adscrawl.net",
  fields: {
    title: {source: "dom", selector: "h1", value: "text", required: true}
  }
)
puts result.dig("data", "title")

inspection = client.spa.inspect(url: "https://www.adscrawl.net")
puts inspection["candidates"]
```

## Remote CDP browsers

```ruby theme={null}
require "adscrawl"

client = AdsCrawl::Client.new
session = client.cdp.create(
  idleTimeoutMs: 600_000,
  maxSessionMs: 3_600_000,
  browserSettings: {viewport: {width: 1440, height: 900}}
)
begin
  version = client.cdp.get_version(session)
  puts version["Browser"]
  # Connect a CDP-compatible Ruby library to session["cdpBaseUrl"].
ensure
  client.cdp.close(session["sessionId"])
end
```

`client.cdp.list` returns active sessions. `get_version` validates the session token URL and deliberately omits the API key. CDP connection URLs contain secrets; do not log them.

## Persistent cloud browsers

```ruby theme={null}
require "adscrawl"

client = AdsCrawl::Client.new
launched = client.cloud_browsers.launch(
  proxy: {server: ENV.fetch("ADSCRAWL_PROXY_SERVER")},
  tabs: ["https://www.adscrawl.net"]
)
browser_id = launched["id"]
begin
  profile = client.cloud_browsers.get(browser_id)
  puts profile.dig("runtime", "status")
ensure
  client.cloud_browsers.stop(browser_id)
end
```

API-key starts and launches require an explicit top-level custom proxy. A `stopping` response means shutdown is pending; poll `get` until `stopped`. A failed launch may expose a cleanup profile ID as `APIError#resource_id`.

## Configuration and errors

```ruby theme={null}
require "adscrawl"

client = AdsCrawl::Client.new(base_url: "https://api.adscrawl.net", timeout_ms: 90_000)
begin
  puts client.markdown(
    {url: "https://www.adscrawl.net", timeoutMs: 60_000},
    timeout_ms: 75_000
  )
rescue AdsCrawl::APIError => error
  puts [error.status, error.api_code, error.trace_id]
rescue AdsCrawl::TimeoutError
  puts "Inspect remote sessions before retrying."
end
```

The API key defaults to `ADSCRAWL_API_KEY`. The base URL defaults to `ADSCRAWL_BASE_URL`, then `ADSCRAWL_API_URL`, then `https://api.adscrawl.net`. Errors include `APIError`, `TimeoutError`, `ConnectionError`, and `ResponseError`. API response bodies are credential-redacted. The HTTP deadline includes response body reading. Requests are not automatically retried.