Machine Learning

Google cloud vision

Learn how to use Google cloud vision with Composio

Overview

SLUG: GOOGLE_CLOUD_VISION

Description

Google Cloud Vision API enables developers to integrate vision detection features into applications, including image labeling, face and landmark detection, optical character recognition (OCR), and explicit content tagging.

Authentication Details

generic_key
stringRequired

Connecting to Google cloud vision

Create an auth config

Use the dashboard to create an auth config for the Google cloud vision toolkit. This allows you to connect multiple Google cloud vision accounts to Composio for agents to use.

1

Select App

Navigate to [Google cloud vision](https://platform.composio.dev?next_page=/marketplace/Google cloud vision).

2

Configure Auth Config Settings

Select among the supported auth schemes of and configure them here.

3

Create and Get auth config ID

Click “Create Google cloud vision Auth Config”. After creation, copy the displayed ID starting with ac_. This is your auth config ID. This is not a sensitive ID — you can save it in environment variables or a database. This ID will be used to create connections to the toolkit for a given user.

Connect Your Account

Using API Key

1from composio import Composio
2
3# Replace these with your actual values
4google_cloud_vision_auth_config_id = "ac_YOUR_GOOGLE_CLOUD_VISION_CONFIG_ID" # Auth config ID created above
5user_id = "0000-0000-0000"  # UUID from database/app
6
7composio = Composio()
8
9def authenticate_toolkit(user_id: str, auth_config_id: str):
10    # Replace this with a method to retrieve an API key from the user.
11    # Or supply your own.
12    user_api_key = input("[!] Enter API key")
13
14    connection_request = composio.connected_accounts.initiate(
15        user_id=user_id,
16        auth_config_id=auth_config_id,
17        config={"auth_scheme": "API_KEY", "val": {"generic_api_key": user_api_key}}
18    )
19
20    # API Key authentication is immediate - no redirect needed
21    print(f"Successfully connected Google cloud vision for user {user_id}")
22    print(f"Connection status: {connection_request.status}")
23    
24    return connection_request.id
25
26
27connection_id = authenticate_toolkit(user_id, google_cloud_vision_auth_config_id)
28
29# You can verify the connection using:
30connected_account = composio.connected_accounts.get(connection_id)
31print(f"Connected account: {connected_account}")

Tools

Executing tools

To prototype you can execute some tools to see the responses and working on the [Google cloud vision toolkit’s playground](https://app.composio.dev/app/Google cloud vision)

Python
1from composio import Composio
2from openai import OpenAI
3import json
4
5openai = OpenAI()
6composio = Composio()
7
8# User ID must be a valid UUID format
9user_id = "0000-0000-0000"  # Replace with actual user UUID from your database
10
11tools = composio.tools.get(user_id=user_id, toolkits=["GOOGLE_CLOUD_VISION"])
12
13print("[!] Tools:")
14print(json.dumps(tools))
15
16def invoke_llm(task = "What can you do?"):
17    completion = openai.chat.completions.create(
18        model="gpt-4o",
19        messages=[
20            {
21                "role": "user",
22                "content": task,  # Your task here!
23            },
24        ],
25        tools=tools,
26    )
27
28    # Handle Result from tool call
29    result = composio.provider.handle_tool_calls(user_id=user_id, response=completion)
30    print(f"[!] Completion: {completion}")
31    print(f"[!] Tool call result: {result}")
32
33invoke_llm()

Tool List

Tool Name: Create Vision Product

Description

Tool to create and return a new Product resource. Use when you need to register a product in a specific project/location after preparing product details.

Action Parameters

description
string
displayName
stringRequired
parent
stringRequired
productCategory
stringRequired
productId
string
productLabels
array

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: Create ReferenceImage

Description

Tool to create a ReferenceImage under a product. Use when adding a new image to a product for detection.

Action Parameters

boundingPolys
array
parent
stringRequired
referenceImageId
string
uri
stringRequired

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: Delete Product

Description

Tool to permanently delete a Product and its reference images. Use after confirming the product's resource name.

Action Parameters

name
stringRequired

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: Get Product

Description

Tool to get information associated with a Product. Use when you have the product resource name and need its details.

Action Parameters

name
stringRequired

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: Get Product Set

Description

Tool to get a ProductSet. Use when you need metadata details of an existing ProductSet by its full resource name. Use after obtaining the resource name.

Action Parameters

name
stringRequired

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: Import Product Sets

Description

Tool to asynchronously import reference images into ProductSets from a CSV in GCS. Use when you need to bulk import images into product sets via a Cloud Storage CSV.

Action Parameters

inputConfig
objectRequired
parent
stringRequired

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: List IndexEndpoints

Description

Tool to list IndexEndpoints in a project and location. Use when you need to retrieve existing IndexEndpoints and handle pagination.

Action Parameters

filter
string
pageSize
integer
pageToken
string
parent
stringRequired

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: List Locations

Description

Tool to list available Vision AI service locations for a project. Use when you need to discover supported regions before making region-specific API calls.

Action Parameters

filter
string
name
stringRequired
pageSize
integer
pageToken
string

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: List Vision API Operations

Description

Tool to list operations that match the specified filter. Use when you need to retrieve all operations under a specific project and location.

Action Parameters

filter
string
name
stringRequired
pageSize
integer
pageToken
string

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: Purge Products

Description

Tool to asynchronously delete products in a ProductSet or orphan products. Use when you need to clean up products at scale; ensure `force` is true to execute.

Action Parameters

deleteOrphanProducts
boolean
force
booleanRequired
parent
stringRequired
productSetPurgeConfig
object

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: Update Product

Description

Tool to update a Product's mutable fields: displayName, description, and productLabels. Use after confirming the product resource name.

Action Parameters

description
string
displayName
string
name
stringRequired
productLabels
array
updateMask
string

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: Update Product Set

Description

Tool to update a ProductSet resource. Use when you need to modify the displayName of an existing ProductSet.

Action Parameters

displayName
stringRequired
name
stringRequired
updateMask
string

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: Add Product to ProductSet

Description

Tool to add a Product to a specified ProductSet. Use after creating both resources in the same project/location to link a product to its set.

Action Parameters

name
stringRequired
product
stringRequired

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: Cancel Vision Operation

Description

Tool to cancel a long-running Vision API operation. Use when you need to abort a pending or in-progress operation.

Action Parameters

name
stringRequired

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: Delete Vision API Operation

Description

Tool to delete a long-running Vision API operation. Use after confirming the operation name.

Action Parameters

name
stringRequired

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: Delete Product Set

Description

Tool to permanently delete a ProductSet. Use after confirming the ProductSet's resource name.

Action Parameters

name
stringRequired

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: Delete Reference Image

Description

Tool to permanently delete a reference image. Use when you have confirmed the reference image's resource name.

Action Parameters

name
stringRequired

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: Get Vision API Operation

Description

Tool to get the latest state of a long-running operation. Use after starting an async Vision API operation to poll its status.

Action Parameters

name
stringRequired

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: Get Reference Image

Description

Tool to get information associated with a ReferenceImage. Use when you have the full resource name and need its metadata.

Action Parameters

name
stringRequired

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: List Products in ProductSet

Description

Tool to list Products in a specified ProductSet. Use when you need to retrieve Products associated with a ProductSet after confirming it exists, with optional pagination.

Action Parameters

name
stringRequired
pageSize
integer
pageToken
string

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: List Projects

Description

Tool to list Google Cloud projects accessible by the authenticated user. Use when you need to enumerate available project IDs and resource names before performing further operations.

Action Parameters

filter
string
pageSize
integer
pageToken
string

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: List Reference Images

Description

Tool to list reference images for a product. Use when you need to retrieve stored reference images under a specified product resource name, with optional pagination.

Action Parameters

pageSize
integer
pageToken
string
parent
stringRequired

Action Response

data
objectRequired
error
string
successful
booleanRequired

Tool Name: Remove Product from ProductSet

Description

Tool to remove a Product from a specified ProductSet. Use after creating both resources in the same project/location to unlink a product from its set.

Action Parameters

name
stringRequired
product
stringRequired

Action Response

data
objectRequired
error
string
successful
booleanRequired