Firecrawl

Learn how to use Firecrawl with Composio

Overview

Enum

FIRECRAWL

Description

Firecrawl automates web crawling and data extraction, enabling organizations to gather content, index sites, and gain insights from online sources at scale

Authentication Details

api_key
stringRequired
base_url
stringDefaults to https://api.firecrawl.dev/v1

Actions

Map multiple url’s based on user options.

Action Parameters

url
stringRequired
search
string
ignoreSitemap
booleanDefaults to True
includeSubdomains
boolean
limit
integerDefaults to 5000

Action Response

data
object
successful
boolean
error

Scrape content from a url and optionally extract structured data using firecrawl’s api.

Action Parameters

url
stringRequired
formats
arrayDefaults to ['markdown']
onlyMainContent
booleanDefaults to True
includeTags
array
excludeTags
array
waitFor
integer
timeout
integerDefaults to 30000
location
object
actions
array
jsonOptions
object

Action Response

data
object
successful
boolean
error

Extract structured data from web pages using firecrawl’s api, then poll until the job completes or fails.

Action Parameters

urls
arrayRequired
prompt
string
schema
object
enable_web_search
boolean

Action Response

data
object
successful
boolean
error

Cancel a crawl job by its uuid.

Action Parameters

id
stringRequired

Action Response

data
object
successful
boolean
error

Get the status of a crawl job by its uuid.

Action Parameters

id
stringRequired

Action Response

data
object
successful
boolean
error

Crawl websites recursively using firecrawl’s api, then poll until the job completes or fails.

Action Parameters

url
stringRequired
excludePaths
array
includePaths
array
maxDepth
integerDefaults to 2
ignoreSitemap
booleanDefaults to True
limit
integerDefaults to 10
allowBackwardLinks
boolean
allowExternalLinks
boolean
webhook
string
scrapeOptions_formats
arrayDefaults to ['markdown']
scrapeOptions_includeTags
array
scrapeOptions_excludeTags
array
scrapeOptions_onlyMainContent
booleanDefaults to True
scrapeOptions_waitFor
integerDefaults to 123

Action Response

data
object
successful
boolean
error