Ai & Machine Learning

Humanloop

Learn how to use Humanloop with Composio

Overview

SLUG: HUMANLOOP

Description

Humanloop helps developers build and refine AI applications, offering user feedback loops, model training, and data annotation to iterate on language model performance

Authentication Details

generic_api_key
stringRequired

Connecting to Humanloop

Create an auth config

Use the dashboard to create an auth config for the Humanloop toolkit. This allows you to connect multiple Humanloop accounts to Composio for agents to use.

1

Select App

Navigate to the Humanloop toolkit page and click “Setup Integration”.

2

Configure Auth Config Settings

Select among the supported auth schemes of and configure them here.

3

Create and Get auth config ID

Click “Create Integration”. After creation, copy the displayed ID starting with ac_. This is your auth config ID. This is not a sensitive ID — you can save it in environment variables or a database. This ID will be used to create connections to the toolkit for a given user.

Connect Your Account

Using API Key

1from composio import Composio
2from composio.types import auth_scheme
3
4# Replace these with your actual values
5humanloop_auth_config_id = "ac_YOUR_HUMANLOOP_CONFIG_ID" # Auth config ID created above
6user_id = "0000-0000-0000"  # UUID from database/app
7
8composio = Composio()
9
10def authenticate_toolkit(user_id: str, auth_config_id: str):
11    # Replace this with a method to retrieve an API key from the user.
12    # Or supply your own.
13    user_api_key = input("[!] Enter API key")
14
15    connection_request = composio.connected_accounts.initiate(
16        user_id=user_id,
17        auth_config_id=auth_config_id,
18        config={"auth_scheme": "API_KEY", "val": user_api_key}
19    )
20
21    # API Key authentication is immediate - no redirect needed
22    print(f"Successfully connected Humanloop for user {user_id}")
23    print(f"Connection status: {connection_request.status}")
24    
25    return connection_request.id
26
27
28connection_id = authenticate_toolkit(user_id, humanloop_auth_config_id)
29
30# You can verify the connection using:
31connected_account = composio.connected_accounts.get(connection_id)
32print(f"Connected account: {connected_account}")

Tools

Executing tools

To prototype you can execute some tools to see the responses and working on the Humanloop toolkit’s playground

Python
1from composio import Composio
2from openai import OpenAI
3import json
4
5openai = OpenAI()
6composio = Composio()
7
8# User ID must be a valid UUID format
9user_id = "0000-0000-0000"  # Replace with actual user UUID from your database
10
11tools = composio.tools.get(user_id=user_id, toolkits=["HUMANLOOP"])
12
13print("[!] Tools:")
14print(json.dumps(tools))
15
16def invoke_llm(task = "What can you do?"):
17    completion = openai.chat.completions.create(
18        model="gpt-4o",
19        messages=[
20            {
21                "role": "user",
22                "content": task,  # Your task here!
23            },
24        ],
25        tools=tools,
26    )
27
28    # Handle Result from tool call
29    result = composio.provider.handle_tool_calls(user_id=user_id, response=completion)
30    print(f"[!] Completion: {completion}")
31    print(f"[!] Tool call result: {result}")
32
33invoke_llm()

Tool List

Tool Name: Create Project

Description

This tool creates a new project in humanloop. it is an independent action that generates a project by accepting a project's name (required), an optional description, and an optional organization id. upon execution, it returns details of the created project, including the project's id, name, description, created at timestamp, and organization id.

Action Parameters

description
string
directory_id
string
name
stringRequired

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: Delete Project

Description

This tool allows you to delete a specific project from your humanloop organization. the deletion is permanent and cannot be undone. all associated data, including sessions, datapoints, and evaluations linked to the project, will be permanently removed.

Action Parameters

project_id
stringRequired

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: List Experiments

Description

This tool retrieves an array of experiments associated with a specific project in humanloop. it requires a project id (starting with 'pr ') and returns details including experiment id, name, description, creation timestamp, status, configuration details, and metrics/results. it is useful for monitoring experiments, analyzing results, tracking model configurations, and comparing experimental setups.

Action Parameters

project_id
stringRequired

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: List Sessions

Description

This tool retrieves a paginated list of sessions for a specific project in humanloop. it requires a project id (and optionally, page and size for pagination) and returns session details such as id, reference id, project information, datapoints count, first inputs, last output, created at, and updated at. this enables users to monitor and analyze historical project interactions.

Action Parameters

page
integer
project_id
stringRequired
size
integer

Action Response

data
objectRequired
error
string
successful
booleanRequired