Brightdata

Learn how to use Brightdata with Composio

Overview

SLUG: BRIGHTDATA

Description

Bright Data provides a comprehensive platform for web data collection, offering tools and services for data extraction, proxy management, and web scraping automation.

Authentication Details

generic_api_key
stringRequired

Connecting to Brightdata

Create an auth config

Use the dashboard to create an auth config for the Brightdata toolkit. This allows you to connect multiple Brightdata accounts to Composio for agents to use.

1

Select App

Navigate to the Brightdata toolkit page and click “Setup Integration”.

2

Configure Auth Config Settings

Select among the supported auth schemes of and configure them here.

3

Create and Get auth config ID

Click “Create Integration”. After creation, copy the displayed ID starting with ac_. This is your auth config ID. This is not a sensitive ID — you can save it in environment variables or a database. This ID will be used to create connections to the toolkit for a given user.

Connect Your Account

Using API Key

1from composio import Composio
2from composio.types import auth_scheme
3
4# Replace these with your actual values
5brightdata_auth_config_id = "ac_YOUR_BRIGHTDATA_CONFIG_ID" # Auth config ID created above
6user_id = "0000-0000-0000" # UUID from database/app
7
8composio = Composio()
9
10def authenticate_toolkit(user_id: str, auth_config_id: str):
11 # Replace this with a method to retrieve an API key from the user.
12 # Or supply your own.
13 user_api_key = input("[!] Enter API key")
14
15 connection_request = composio.connected_accounts.initiate(
16 user_id=user_id,
17 auth_config_id=auth_config_id,
18 config={"auth_scheme": "API_KEY", "val": user_api_key}
19 )
20
21 # API Key authentication is immediate - no redirect needed
22 print(f"Successfully connected Brightdata for user {user_id}")
23 print(f"Connection status: {connection_request.status}")
24
25 return connection_request.id
26
27
28connection_id = authenticate_toolkit(user_id, brightdata_auth_config_id)
29
30# You can verify the connection using:
31connected_account = composio.connected_accounts.get(connection_id)
32print(f"Connected account: {connected_account}")

Tools

Executing tools

To prototype you can execute some tools to see the responses and working on the Brightdata toolkit’s playground

Python
1from composio import Composio
2from openai import OpenAI
3import json
4
5openai = OpenAI()
6composio = Composio()
7
8# User ID must be a valid UUID format
9user_id = "0000-0000-0000" # Replace with actual user UUID from your database
10
11tools = composio.tools.get(user_id=user_id, toolkits=["BRIGHTDATA"])
12
13print("[!] Tools:")
14print(json.dumps(tools))
15
16def invoke_llm(task = "What can you do?"):
17 completion = openai.chat.completions.create(
18 model="gpt-4o",
19 messages=[
20 {
21 "role": "user",
22 "content": task, # Your task here!
23 },
24 ],
25 tools=tools,
26 )
27
28 # Handle Result from tool call
29 result = composio.provider.handle_tool_calls(user_id=user_id, response=completion)
30 print(f"[!] Completion: {completion}")
31 print(f"[!] Tool call result: {result}")
32
33invoke_llm()

Tool List

Tool Name: Crawl API

Description

Tool to trigger a site crawl job to extract content across multiple pages or entire domains. use when you need to start a crawl for a given dataset and list of urls.

Action Parameters

custom_output_fields
string
dataset_id
stringRequired
include_errors
boolean
items
arrayRequired

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: Dataset List

Description

Tool to list all dataset ids of all scraper apis. use when you need to retrieve identifiers of all available marketplace datasets.

Action Parameters

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: Filter Dataset

Description

Tool to apply custom filter criteria to a marketplace dataset (beta). use after selecting a dataset to generate a filtered snapshot.

Action Parameters

dataset_id
stringRequired
files
array
filter
objectRequired
records_limit
integer

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: Get Available Cities

Description

Tool to get available static network cities for a given country. use when you need to configure static proxy endpoints after selecting a country.

Action Parameters

country
stringRequired
pool_ip_type
stringDefaults to dc

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: Get Available Countries

Description

Tool to list available countries and their iso 3166-1 alpha-2 codes. use when you need to configure zones with valid country codes before provisioning proxies.

Action Parameters

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: List Web Unlocker Zones

Description

Tool to list web unlocker api zones. use when you need to retrieve all configured zones for your bright data account after authentication.

Action Parameters

Action Response

data
objectRequired
error
string
successful
booleanRequired