---
canonical_url: "https://docs.composio.dev/toolkits/spider"
markdown_url: "https://docs.composio.dev/toolkits/spider.md"
product: "platform"
version: null
last_updated: null
---

# Spider

Spider provides APIs for web crawling, scraping, search, screenshots, content transformation, AI extraction, and high-concurrency data collection.

- **Category:** web scraping & browser
- **Auth:** API_KEY
- **Composio-managed OAuth available?** N/A
- **Tools:** 15
- **Triggers:** 0
- **Slug:** `SPIDER`
- **Version:** 00000000_00

## Tools

### AI Browser

**Slug:** `SPIDER_AI_BROWSER`

Drive a browser from a natural-language prompt and extract results. Requires AI Studio. Prompts can click, navigate, submit forms, purchase, message, or change accounts, so request those external effects only when intended and authorized.

#### Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `url` | string | Yes | Initial page URL for the browser session. |
| `limit` | integer | No | Maximum pages or browser results to process; keep bounded to control AI cost. |
| `prompt` | string | Yes | Browser instruction. State when the task is read-only. Clicks, form submissions, purchases, messages, and account changes can cause external side effects and should be requested only when intended and authorized. |
| `metadata` | boolean | No | Include page metadata. Structured output is returned in metadata.extracted_data regardless. |
| `wait_for` | object | No | Bounded selector, DOM, network, delay, and navigation wait conditions. |
| `return_format` | string ("raw" | "markdown" | "commonmark" | "text" | "html2text" | "xml" | "bytes" | "screenshot" | "empty") | No | Page content format returned after browser interaction. |
| `advanced_options` | object | No | Browser rendering and output, proxy and geo, session and cache, budget, wait, and event-tracking controls. Raw code and automation scripts are not accepted; express browser actions through prompt. |
| `extraction_schema` | object | No | Optional contract that pins the structured result shape. |

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |

### AI Crawl Website

**Slug:** `SPIDER_AI_CRAWL`

Crawl pages and use an AI prompt to extract structured data. Requires AI Studio. limit defaults to 1 to bound crawl and model costs; depth 0 removes the depth limit.

#### Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `url` | string | Yes | Starting URL for the prompt-driven crawl. |
| `depth` | integer | No | Maximum link depth from the starting URL; 0 removes the depth limit, so use limit to bound the crawl. |
| `limit` | integer | No | Maximum pages to process; 0 removes the page limit and processes all pages found. Each page can add crawl and AI cost. |
| `prompt` | string | Yes | Precise natural-language instruction describing what to find and extract. |
| `metadata` | boolean | No | Include page metadata. Structured extracted_data is returned either way. |
| `return_format` | string | No | Page content format or formats to include with the extracted result. |
| `cleaning_intent` | string ("extraction" | "action" | "general") | No | How Spider should reduce page HTML before the model reads it. |
| `advanced_options` | object | No | Additional typed crawl controls for traversal, browser behavior, output, proxy and geo routing, sessions, caching, and budgets. Use the named request fields for the AI objective, result shape, and output format. |
| `extraction_schema` | object | No | Optional contract that fixes the shape of each structured result. |

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |

### AI Discover Links

**Slug:** `SPIDER_AI_DISCOVER_LINKS`

Collect links matching a natural-language prompt and return structured context. Requires AI Studio. limit defaults to 1 to bound crawl and model costs; depth 0 removes the depth limit.

#### Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `url` | string | Yes | Starting URL for prompt-guided link collection. |
| `depth` | integer | No | Maximum traversal depth; 0 removes the depth limit, so use limit to bound discovery. |
| `limit` | integer | No | Maximum pages to inspect; 0 removes the page limit and inspects all pages found. Each page can add crawl and AI cost. |
| `prompt` | string | Yes | Describe which links to collect and any structured context to extract. |
| `metadata` | boolean | No | Include page metadata. Structured extracted_data is returned either way. |
| `return_format` | string | No | Content format or formats used for extraction. |
| `cleaning_intent` | string ("extraction" | "action" | "general") | No | How Spider should reduce page content before the model reads it. |
| `advanced_options` | object | No | Additional traversal and page-processing options for AI link discovery. |
| `extraction_schema` | object | No | Optional JSON Schema contract for the structured link context. |

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |

### AI Scrape Page

**Slug:** `SPIDER_AI_SCRAPE`

Fetch exact pages and use an AI prompt to extract structured data. Requires AI Studio; batched or complex extraction increases model cost.

#### Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `url` | string | Yes | Exact page URL to fetch for prompt-driven extraction. |
| `limit` | integer | No | Maximum items or pages to process. Keep at 1 for one URL to bound AI cost. |
| `prompt` | string | Yes | Precise extraction instruction describing the desired result. |
| `metadata` | boolean | No | Include page metadata. Structured output is returned in metadata.extracted_data regardless. |
| `return_format` | string | No | Page content format supplied to or returned alongside extraction. |
| `cleaning_intent` | string ("extraction" | "action" | "general") | No | How Spider should clean page content before the model reads it. |
| `advanced_options` | object | No | Browser readiness, rendering, output filtering, proxy and geo, session, cache, and budget controls. Use named request fields for AI extraction and output format instead of repeating them here. |
| `extraction_schema` | object | No | Optional contract that pins the structured result shape. |

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |

### AI Search Web

**Slug:** `SPIDER_AI_SEARCH`

Run prompt-driven web search with optional page fetching and structured extraction. Requires AI Studio and defaults to one result without page fetching.

#### Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `limit` | integer | No | Maximum search results or fetched pages returned for the AI task to process. |
| `prompt` | string | Yes | Natural-language research or extraction objective. |
| `search` | string | No | Literal search query. Omit it to let Spider generate a query from the prompt. |
| `metadata` | boolean | No | Include page metadata when search-result pages are fetched. |
| `result_count` | integer | No | Number of web search results to request. The AI task may process fewer when limit is lower. |
| `return_format` | string ("raw" | "markdown" | "commonmark" | "text" | "html2text" | "xml" | "bytes" | "screenshot" | "empty") | No | Content format used when search-result pages are fetched. |
| `extraction_schema` | object | No | Optional contract that fixes the structured result shape. |
| `fetch_page_content` | boolean | No | Fetch result pages for extraction. Enabling this increases latency, payload size, and cost. |
| `standard_search_parameters` | object | No | Search localization, geographic targeting, filters, speed, and pagination controls. |

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |

### Capture Screenshot

**Slug:** `SPIDER_CAPTURE_SCREENSHOT`

Render one page and capture a PNG, JPEG, or WebP screenshot with a provider-side response-content cap. Full-page capture is the documented default; returns compact metadata by default, optional base64 content, or an offloaded file in binary mode.

#### Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `url` | string | Yes | Exactly one page URL. Full-page capture is enabled by default and bounded by the provider response-content cap. |
| `fast` | boolean | No | Use Spider's speed-optimized screenshot renderer. |
| `binary` | boolean | No | Return the captured image as an offloaded file instead of JSON metadata. |
| `full_page` | boolean | No | Capture the full scrollable page; false captures the viewport only. |
| `cdp_params` | object | No | Chrome capture format, quality, clip, and surface settings. |
| `screenshot` | boolean | No | Explicitly enable screenshot capture. |
| `block_images` | boolean | No | Block page images before capture. |
| `omit_background` | boolean | No | Use a transparent background when the image format supports it. |
| `advanced_options` | object | No | Browser rendering, proxy and geo, cache, network, and output-filtering controls. |
| `include_image_data` | boolean | No | Include base64 image content in JSON output. Leave false for metadata and content length only. |

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |

### Crawl Website

**Slug:** `SPIDER_CRAWL`

Crawl pages from one or more starting URLs. Defaults to at most 10 pages and depth 1; raising breadth increases cost and response size. Set billing_mode='unlimited' only for an active Unlimited plan with purchased concurrency.

#### Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `url` | string | Yes | Starting URL, or comma-separated starting URLs. |
| `depth` | integer | No | Maximum link depth from each starting URL; 0 removes the depth limit, so use limit to bound the crawl. |
| `limit` | integer | No | Maximum pages to process; 0 removes the page limit and processes all pages found, subject to credits and other crawl bounds. |
| `metadata` | boolean | No | Whether to include page metadata. |
| `billing_mode` | string ("payg" | "unlimited") | No | Model-only billing selector. Defaults to PAYG, which uses credits. Unlimited requires an active Unlimited plan, uses purchased concurrency, rejects AI extraction fields, and may return 429 immediately when all purchased concurrency is active. |
| `return_format` | string | No | Content format or formats Spider should return. |
| `advanced_options` | object | No | Traversal filters, browser readiness and rendering, output processing, proxy and geo, session, cache, network, and budget controls. Named request fields define the crawl bounds and output format. |

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |

### Discover Links

**Slug:** `SPIDER_DISCOVER_LINKS`

Discover URLs from a site without returning full page content. Defaults to 25 pages and depth 1; larger traversal costs more. Set billing_mode='unlimited' only for an active Unlimited plan with purchased concurrency.

#### Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `url` | string | Yes | Starting URL for link discovery. |
| `depth` | integer | No | Maximum traversal depth; 0 removes the depth limit, so use limit to bound discovery. |
| `limit` | integer | No | Maximum pages to inspect; 0 removes the page limit and inspects all pages found. |
| `billing_mode` | string ("payg" | "unlimited") | No | Model-only billing selector. Defaults to PAYG, which uses credits. Unlimited requires an active Unlimited plan, uses purchased concurrency, rejects AI extraction fields, and may return 429 immediately when all purchased concurrency is active. |
| `advanced_options` | object | No | Traversal filters, browser readiness and rendering, output processing, proxy and geo, session, cache, network, and budget controls. Named request fields define discovery bounds. |

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |

### Fetch Configured Page

**Slug:** `SPIDER_FETCH_CONFIGURED_PAGE`

Fetch a domain and path with Spider's saved scraper configuration, discovering and storing one on first use. Defaults to one page; first-time discovery can take 30-120 seconds and incur AI cost.

#### Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `path` | string | Yes | Site path. Use '/' for the root; slash-separated segments are preserved safely. |
| `limit` | integer | No | Pages to fetch. Values above 1 crawl up to this many pages and increase cost. |
| `domain` | string | Yes | Bare domain name without a URL scheme, port, or path. |
| `return_format` | string | No | Content format or formats Spider should return. |
| `advanced_options` | object | No | Browser readiness and rendering, output processing, proxy and geo, session, cache, network, and budget overrides. The named domain, path, page limit, and output format remain authoritative; AI extraction is unavailable. |

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |

### Get Credit Balance

**Slug:** `SPIDER_GET_CREDIT_BALANCE`

Get the connected Spider account's remaining credit balance and documented record metadata.

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |

### List Crawl Logs

**Slug:** `SPIDER_LIST_CRAWL_LOGS`

Return the connected account's crawl and scrape activity from the last 24 hours.

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |

### List Scraper Directory

**Slug:** `SPIDER_LIST_SCRAPER_DIRECTORY`

Browse public curated scraper configurations with domain, category, text, and page-number filters.

#### Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `page` | integer | No | Page number to return, starting at 1. |
| `limit` | integer | No | Maximum number of directory entries to return on this page. |
| `domain` | string | No | Filter scraper configurations by domain. |
| `search` | string | No | Search scraper directory text and metadata. |
| `category` | string | No | Filter scraper configurations by category. |

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |

### Scrape Pages

**Slug:** `SPIDER_SCRAPE`

Fetch and extract exact pages without following links. Use crawl only for traversal; set billing_mode='unlimited' only for an active Unlimited plan with purchased concurrency.

#### Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `url` | string | Yes | Page URL, comma-separated URLs, or Spider's documented batched request value. |
| `metadata` | boolean | No | Whether to include page metadata. |
| `billing_mode` | string ("payg" | "unlimited") | No | Model-only billing selector. Defaults to PAYG, which uses credits. Unlimited requires an active Unlimited plan, uses purchased concurrency, rejects AI extraction fields, and may return 429 immediately when all purchased concurrency is active. |
| `return_format` | string | No | Content format or formats Spider should return. |
| `advanced_options` | object | No | Browser readiness and rendering, output processing, proxy and geo, session, cache, network, and budget controls. Named request fields define the target pages and output format. |

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |

### Search Web

**Slug:** `SPIDER_SEARCH_WEB`

Search the web and optionally fetch result pages. Defaults to 10 results without page fetching or automatic multi-page expansion.

#### Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `num` | integer | No | Requested number of search results. |
| `tbs` | string | No | Search filter expression, such as qdr:d for the past day. |
| `page` | integer | No | Search-results page number. |
| `radius` | integer | No | Geographic search radius in meters, used with coordinates. |
| `search` | string | Yes | Search query; Spider names this field search, not query. |
| `country` | string | No | Country code or provider-supported country selector for localized results. |
| `language` | string | No | Language selector for search results. |
| `latitude` | number | No | Latitude for geographic search targeting; use with longitude. |
| `location` | string | No | Human-readable search location. |
| `longitude` | number | No | Longitude for geographic search targeting; use with latitude. |
| `quick_search` | boolean | No | Use Spider's faster search path. |
| `search_limit` | integer | No | Maximum search results Spider may process; 0 processes all results. |
| `auto_pagination` | boolean | No | Fetch additional search pages to reach the requested count; this adds requests and cost. |
| `advanced_options` | object | No | Browser readiness and rendering, output processing, proxy and geo, session, cache, network, and budget controls used when fetching result pages. Named search fields define the query, count, and localization. |
| `fetch_page_content` | boolean | No | Fetch result pages as well as snippets; this increases latency, response size, and cost. |

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |

### Transform Content

**Slug:** `SPIDER_TRANSFORM_CONTENT`

Convert supplied HTML to markdown, text, or cleaned output without fetching a URL; the total serialized JSON request must not exceed 10 MB.

#### Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | array | Yes | Content items to transform. The total serialized JSON request must not exceed 10 MB. |
| `clean` | boolean | No | Remove footers, navigation, and similar noise for AI-oriented output. |
| `clean_full` | boolean | No | Apply Spider's full cleaning behavior, including stripping unwanted attributes. |
| `readability` | boolean | No | Apply readability preprocessing to remove page boilerplate before conversion. |
| `return_format` | string | No | Output format or formats for the transformed content. |

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |
