Tool Calling and Composio

Tool calling enables AI models to perform tasks beyond simple conversations, allowing them to interact with external services and applications. Instead of just answering questions, your AI assistant can now browse the internet, schedule meetings, update CRM records, or even manage tasks in project management tools.

With Composio, your AI apps and agents gain access to over 250 integrations, including popular services like GitHub, Google Calendar, Salesforce, Slack, and many more. This means your AI agents can seamlessly handle real-world tasks, saving you time and effort.

Overview

Tool calling flow

Here’s the above example in code:

1 from composio_openai import ComposioToolSet, action
2 from typing import Annotated
3 from openai import OpenAI
4 import math
5 
6 client = OpenAI()
7 toolset = ComposioToolSet()
8 
9 @action(toolname="calculate_square_root", requires=["math"])
10 def calculate_square_root(
11     a: Annotated[int, "Number from which to take the square root"],
12 ) -> float:
13     """
14     Calculate the square root of a number.
15     :param a: Number from which to take the square root
16     :return sqrt: Square root of the number
17     """
18     return math.sqrt(a)
19 
20 tools = toolset.get_tools([calculate_square_root])
21 
22 question = "What is the square root of 212?"
23 
24 response = client.chat.completions.create(
25     model="gpt-4o-mini",
26     messages=[
27         {
28             "role": "user",
29             "content": question,
30         }
31     ],
32     tools=tools,
33     tool_choice="auto",
34 )
35 
36 result = toolset.handle_tool_calls(response=response)
37 
38 print("Question: ", question)
39 print("Answer: ", result[0]["data"]["sum"])

Create tool

Convert a function into LLM-readable form using the @action wrapper (Python) or the callback function (JS)

LLM calls the tool

The LLM reasons about the user’s query and decides whethere to use a tool.
If yes, the LLM generate a properly formatted tool use request with the input parameters specified.

Handling of the tool call

Composio intercepts, interprets, and calls the actual method defined. handle_tool_calls method interprets the tool call and calls the actual method defined.

Tool Calling with Composio

Composio supports three main ways to use tool calling:

Hosted Tools

Pre-built tools that Composio hosts and maintains, giving you instant access to thousands of actions across hundreds of apps.

Local Tools

Tools that run locally in your environment, like file operations or custom business logic.

Custom Tools

Your own tools defined using Composio’s tool definition format, which can be hosted anywhere.

Using Composio’s Hosted Tools

Composio hosts a growing list of tools from various popular apps like Gmail, Notion to essential apps for AI Engineers like Firecrawl, Browserbase.

This lets you build AI apps and agents without having to manually write the API calls and integrations in the tool format.

Here’s an example of using Firecrawl with to scrape a webpage.

You will need to add a Firecrawl integration. Learn how to do it here

1 from composio_openai import ComposioToolSet
2 from openai import OpenAI
3 
4 client = OpenAI()
5 toolset = ComposioToolSet()
6 
7 tools = toolset.get_tools(["FIRECRAWL_SCRAPE_EXTRACT_DATA_LLM"])
8 task = "Scrape https://example.com and extract the data"
9 response = client.chat.completions.create(
10     model="gpt-4o-mini",
11     messages=[{"role": "user", "content": task}],
12     tools=tools,
13     tool_choice="auto",
14 )
15 
16 result = toolset.handle_tool_calls(response)
17 print(result)

Using Composio’s Local Tools

Composio ships with a host of tools that run locally on your system for performing common development tasks like file operations, shell commands, and code execution.

These don’t require any integration or authentication.

Local tools are currently only supported on our Python SDK

These tools run directly on the defined workspace while maintaining security through permission controls.

Workspaces

Workspaces are environments where local tools are fun. Read more about them here.

Workspaces?

Workspaces are environments where local tools run separately from your system.

1 from composio_claude import ComposioToolSet, action
2 from anthropic import Anthropic
3 
4 client = Anthropic()
5 
6 toolset = ComposioToolSet()
7 tools = toolset.get_tools(["FILETOOL_LIST_FILES"])
8 
9 question = "List all files in the current directory"
10 
11 response = client.messages.create(
12     model="claude-3-5-haiku-latest",
13     max_tokens=1024,
14     tools=tools,
15     messages=[{"role": "user", "content": question}],
16 )
17 result = toolset.handle_tool_calls(response)
18 print(result)

Using Custom Tools

Custom tools allow you to define your own functions for LLMs to call without manually writing JSON schemas. This provides a unified tool calling experience across your application.

These can be:

Functions you define in your codebase
External APIs you want to expose to the LLM
Business logic specific to your application

Custom tools can be seamlessly combined with both local and hosted tools

For creating custom tools using OpenAPI specifications, visit the Custom Tools Dashboard where you can upload and manage your API specs.

Here’s how to create and use custom tools with Composio:

1 from composio_openai import ComposioToolSet, action
2 from typing import Annotated
3 from openai import OpenAI
4 import math
5 
6 toolset = ComposioToolSet()
7 client = OpenAI()
8 
9 # Create a custom tool
10 @action(toolname="math", requires=["math"])
11 def calculate_sum(
12     a: Annotated[int, "First number"],
13     b: Annotated[int, "Second number"],
14 ) -> int:
15     """
16     Calculate the sum of two numbers.
17     :param a: First number
18     :param b: Second number
19     :return sum: Sum of the two numbers
20     """
21     return a + b
22 
23 tools = toolset.get_tools([calculate_sum])
24 
25 question = "What is 3932 + 2193?"
26 
27 response = client.chat.completions.create(
28     model="gpt-4o-mini",
29     messages=[{"role": "user", "content": question}],
30     tools=tools,
31     tool_choice="auto",
32 )
33 
34 result = toolset.handle_tool_calls(response=response)
35 print(result)

1	from composio_openai import ComposioToolSet, action
2	from typing import Annotated
3	from openai import OpenAI
4	import math
5
6	client = OpenAI()
7	toolset = ComposioToolSet()
8
9	@action(toolname="calculate_square_root", requires=["math"])
10	def calculate_square_root(
11	a: Annotated[int, "Number from which to take the square root"],
12	) -> float:
13	"""
14	Calculate the square root of a number.
15	:param a: Number from which to take the square root
16	:return sqrt: Square root of the number
17	"""
18	return math.sqrt(a)
19
20	tools = toolset.get_tools([calculate_square_root])
21
22	question = "What is the square root of 212?"
23
24	response = client.chat.completions.create(
25	model="gpt-4o-mini",
26	messages=[
27	{
28	"role": "user",
29	"content": question,
30	}
31	],
32	tools=tools,
33	tool_choice="auto",
34	)
35
36	result = toolset.handle_tool_calls(response=response)
37
38	print("Question: ", question)
39	print("Answer: ", result[0]["data"]["sum"])

1	from composio_claude import ComposioToolSet, action
2	from anthropic import Anthropic
3
4	client = Anthropic()
5
6	toolset = ComposioToolSet()
7	tools = toolset.get_tools(["FILETOOL_LIST_FILES"])
8
9	question = "List all files in the current directory"
10
11	response = client.messages.create(
12	model="claude-3-5-haiku-latest",
13	max_tokens=1024,
14	tools=tools,
15	messages=[{"role": "user", "content": question}],
16	)
17	result = toolset.handle_tool_calls(response)
18	print(result)