Firecrawl

Learn how to use Firecrawl with Composio

Overview

Enum

FIRECRAWL

Description

Firecrawl automates web crawling and data extraction, enabling organizations to gather content, index sites, and gain insights from online sources at scale

Authentication Details

api_key
stringRequired
base_url
stringDefaults to https://api.firecrawl.dev/v1

Actions

Cancel a crawl job by its uuid.

Action Parameters

id
stringRequired

Action Response

data
object
error
successful
boolean

Get the status of a crawl job by its uuid.

Action Parameters

id
stringRequired

Action Response

data
object
error
successful
boolean

Crawl websites recursively using firecrawl’s api, then poll until the job completes or fails.

Action Parameters

allowBackwardLinks
boolean
allowExternalLinks
boolean
excludePaths
array
ignoreSitemap
booleanDefaults to True
includePaths
array
limit
integerDefaults to 10
maxDepth
integerDefaults to 2
scrapeOptions_excludeTags
array
scrapeOptions_formats
arrayDefaults to ['markdown']
scrapeOptions_includeTags
array
scrapeOptions_onlyMainContent
booleanDefaults to True
scrapeOptions_waitFor
integerDefaults to 123
url
stringRequired
webhook
string

Action Response

data
object
error
successful
boolean

Extract structured data from web pages using firecrawl’s api, then poll until the job completes or fails.

Action Parameters

enable_web_search
boolean
prompt
string
schema
object
urls
arrayRequired

Action Response

data
object
error
successful
boolean

Map multiple url’s based on user options.

Action Parameters

ignoreSitemap
booleanDefaults to True
includeSubdomains
boolean
limit
integerDefaults to 5000
search
string
url
stringRequired

Action Response

data
object
error
successful
boolean

Scrape content from a url and optionally extract structured data using firecrawl’s api.

Action Parameters

actions
array
excludeTags
array
formats
arrayDefaults to ['markdown']
includeTags
array
jsonOptions
object
location
object
onlyMainContent
booleanDefaults to True
timeout
integerDefaults to 30000
url
stringRequired
waitFor
integer

Action Response

data
object
error
successful
boolean