---
canonical_url: "https://docs.composio.dev/toolkits/llm_api"
markdown_url: "https://docs.composio.dev/toolkits/llm_api.md"
product: "platform"
version: null
last_updated: null
---

# LLM API

LLM API provides a unified inference API for text, embeddings, images, audio, video, files, and batch workloads across multiple AI providers.

- **Category:** artificial intelligence
- **Auth:** API_KEY
- **Composio-managed OAuth available?** N/A
- **Tools:** 12
- **Triggers:** 0
- **Slug:** `LLM_API`
- **Version:** 00000000_00

## Tools

### Cancel Batch

**Slug:** `LLM_API_CANCEL_BATCH`

Request best-effort cancellation of an in-progress batch. Return the provider's actual batch state without implying cancellation took effect; work may still finish and incur charges before the request is applied.

#### Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `batch_id` | string | Yes | ID of the batch whose cancellation should be requested. |

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |

### Count Anthropic Message Tokens

**Slug:** `LLM_API_COUNT_ANTHROPIC_MESSAGE_TOKENS`

Count input tokens for an Anthropic-compatible message request without generating a model response.

#### Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `model` | string | Yes | Model ID or alias whose tokenizer and message framing should be used. |
| `tools` | array | No | Anthropic-compatible tool definitions whose framing should be included in the count. |
| `system` | string | No | System prompt as text or Anthropic-compatible content blocks to include in the count. |
| `messages` | array | Yes | Ordered, non-empty Anthropic-compatible messages to include in the token count. |
| `tool_choice` | object | No | Anthropic-compatible tool-selection configuration to include in the message framing. |

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |

### Create Anthropic Message

**Slug:** `LLM_API_CREATE_ANTHROPIC_MESSAGE`

Generate one complete, non-streaming Anthropic-compatible message with flexible content blocks, system prompts, and tool use.

#### Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `model` | string | Yes | Claude-family or other model ID exposed through the Anthropic-compatible Messages contract. |
| `tools` | array | No | Anthropic-compatible tool definitions available to the model. |
| `top_p` | number | No | Nucleus-sampling probability from 0 through 1, when supported. |
| `system` | string | No | Anthropic-compatible system prompt as text or supported content blocks. |
| `messages` | array | Yes | Ordered, non-empty Anthropic-compatible conversation messages. |
| `max_tokens` | integer | Yes | Maximum number of output tokens to generate. |
| `temperature` | number | No | Sampling temperature from 0 through 1, when supported by the model. |
| `tool_choice` | object | No | Anthropic-compatible tool-selection configuration. |
| `stop_sequences` | array | No | Custom text sequences that stop generation. |

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |

### Create Chat Completion

**Slug:** `LLM_API_CREATE_CHAT_COMPLETION`

Generate a complete, non-streaming OpenAI-compatible chat response from role-based messages, optionally using tools, multimodal content, structured output, reasoning, or web search supported by the selected model.

#### Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `model` | string | Yes | Model ID or alias from Discover Models that supports chat completions. |
| `tools` | array | No | OpenAI-compatible function or provider tool definitions available to the model. |
| `top_p` | number | No | Nucleus-sampling probability accepted by the selected model; supported ranges are model-dependent. |
| `messages` | array | Yes | Ordered OpenAI-compatible conversation messages with model-supported content and tool-call fields. |
| `max_tokens` | integer | No | Maximum tokens to generate for models using the legacy max_tokens field. |
| `web_search` | boolean | No | Enable provider web search when the selected model and project support it. |
| `temperature` | number | No | Sampling temperature accepted by the selected model; supported ranges are model-dependent. |
| `tool_choice` | string | No | How the model may select supplied tools, using the OpenAI-compatible form supported by the model. |
| `no_reasoning` | boolean | No | Disable reasoning when supported, including non-reasoning selection for auto routing. |
| `response_format` | object | No | OpenAI-compatible response-format object for text, JSON, or a model-supported JSON schema. |
| `reasoning_effort` | string | No | Reasoning effort for models that advertise this parameter. |
| `web_search_options` | object | No | Provider web-search options to apply when web search is enabled. |
| `max_completion_tokens` | integer | No | Maximum completion tokens for models using the newer field. Avoid setting both token limits unless the model supports them. |

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |

### Create Embeddings

**Slug:** `LLM_API_CREATE_EMBEDDINGS`

Create vector embeddings for one text or a batch of texts with a selected embedding model.

#### Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `user` | string | No | Stable, non-sensitive end-user identifier for provider abuse monitoring. |
| `input` | string | Yes | One text or a non-empty list of texts to embed in one request. |
| `model` | string | Yes | Embedding model ID or alias from the model catalog. |
| `dimensions` | integer | No | Requested vector length when the selected model supports configurable dimensions. |
| `encoding_format` | string ("float" | "base64") | No | Return each embedding as numeric floats or a base64-encoded string. |

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |

### Create Response

**Slug:** `LLM_API_CREATE_RESPONSE`

Generate one complete, non-streaming OpenAI Responses-compatible result from text or structured input, preserving reasoning and tool items.

#### Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `text` | object | No | Responses text-output configuration supported by the selected model. |
| `input` | string | Yes | Text or OpenAI Responses-compatible input messages/items, including model-supported multimodal content. |
| `model` | string | Yes | Model ID or alias that supports the Responses API binding. |
| `tools` | array | No | Responses-compatible tools available to the model. |
| `reasoning` | object | No | Reasoning configuration, such as an effort level, for supported models. |
| `tool_choice` | string | No | Responses-compatible tool-selection policy. |
| `instructions` | string | No | High-level instructions applied to the response. |
| `response_format` | object | No | Compatibility response-format object supported by the selected model. |
| `max_output_tokens` | integer | No | Maximum output tokens. The provider requires at least 16 when supplied. |

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |

### Delete File

**Slug:** `LLM_API_DELETE_FILE`

Permanently delete a caller-owned or explicitly authorized project file. This destructive operation cannot be undone; a missing file fails with not found rather than reporting a successful deletion.

#### Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `file_id` | string | Yes | ID of a caller-owned or explicitly authorized project file to delete permanently. This destructive operation cannot be undone. |

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |

### Discover Models

**Slug:** `LLM_API_DISCOVER_MODELS`

Return one model's full live capabilities by ID, or browse a compact, bounded view of the live model catalog using local capability filters.

#### Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `kind` | string | No | Optional exact catalog kind to retain, such as chat or embedding. |
| `model` | string | No | Exact model ID or alias to inspect; omit to browse the catalog. |
| `free_only` | boolean | No | When browsing, return only models cataloged as free. This does not guarantee that the connected account can invoke them. |
| `max_results` | integer | No | Maximum matching catalog entries to return after local filtering (1-200). |
| `exclude_deprecated` | boolean | No | Exclude deprecated models from catalog browsing. |
| `include_deactivated` | boolean | No | Include deactivated models in catalog browsing. |
| `required_capability` | string | No | Optional capability or supported parameter that returned models must advertise, such as tools or image_input. |

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |

### Get Account and Credits

**Slug:** `LLM_API_GET_ACCOUNT_AND_CREDITS`

Return the connected user, project, organization, purchased and bonus credit balances, funding mode, and balance gating in one account-status read.

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |

### List or Get Batches

**Slug:** `LLM_API_LIST_OR_GET_BATCHES`

Return one asynchronous batch and its latest lifecycle, token, and cost status by ID, or one provider-default page of project batches when no batch ID is supplied. List mode supports provider cursor pagination.

#### Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `after` | string | No | In list mode, return the page after this batch ID. Pass the prior response's last_id to continue. |
| `limit` | integer | No | In list mode, maximum number of batches for the provider to return. |
| `batch_id` | string | No | Exact batch ID whose current status should be returned; omit to return one provider-default page of batches. |

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |

### List or Get Files

**Slug:** `LLM_API_LIST_OR_GET_FILES`

Return metadata for one project file by ID, or one provider-default page of project file metadata when no file ID is supplied. List mode supports purpose filtering and provider cursor pagination.

#### Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `after` | string | No | In list mode, return the page after this file ID. When has_more is true, pass the final files[].id from the previous response. |
| `limit` | integer | No | In list mode, maximum number of files for the provider to return. |
| `file_id` | string | No | Exact file ID to inspect; omit to return one provider-default page of project files. |
| `purpose` | string | No | In list mode, return only files uploaded for this purpose. |

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |

### Submit Batch

**Slug:** `LLM_API_SUBMIT_BATCH`

Create an asynchronous batch from an existing purpose=batch file ID, or upload a JSONL input file with purpose=batch and create the batch in one call.

#### Input Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `endpoint` | string ("/v1/chat/completions" | "/v1/responses") | Yes | Endpoint used by every JSONL request in this batch. |
| `input_file` | object | No | JSONL batch request file to upload before submission. |
| `input_file_id` | string | No | Existing purpose=batch file ID. Provide exactly one of input_file_id or input_file. |
| `completion_window` | string | No | Provider completion window. The only documented value is 24h. |

#### Output

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `data` | string | Yes | Data from the action execution |
| `error` | string | No | Error if any occurred during the execution of the action |
| `successful` | boolean | Yes | Whether or not the action execution was successful or not |