Skip to content
Theme
OnDevice Tools

AIAI Tools & Apps

Gemini API Tool Retrieval helps agents manage hundreds of tools

Google says Tool Retrieval is in preview in the Gemini Interactions API. It lets agents keep large tool libraries available and load schema details only when needed.

Illustration of a Gemini API dashboard loading tool schemas on demand for an AI agent

Key takeaways

  • Tool Retrieval is in preview in the Gemini Interactions API.
  • It loads tool schemas on demand to reduce token use and delay.
  • Developers enable it with tool_search and defer_loading.
  • It supports Gemini 3.6, later Gemini models and Gemini 4.
In this article
  1. What Tool Retrieval changes
  2. How developers enable it
  3. Caching, debugging and costs
  4. Models and practical impact

What Tool Retrieval changes

AI agents often use many tools, such as functions for refunds, searches or account changes. A parameter schema describes the inputs a tool accepts. Sending the full schema for every tool with each request can increase token use, add delay and make tool selection less accurate as the list grows. Tokens are the units of text that a model processes, and latency is the time taken to receive a response.

Tool Retrieval changes this by allowing an agent to retain the whole tool library without sending every schema each time. The model can still see each deferred tool's name and short description, while the detailed schema remains outside the prompt until required. If the task needs a tool, the model requests it by name, the API supplies the schema and the function call, which is the model's request to use a tool, can happen in the same interaction.

Google handles the search on its own systems, so developers do not need to build or host an index. Teams can keep frequently used tools loaded immediately and defer less common ones. The interaction record can show the search, schema load and function call steps, which may help when checking tool names or descriptions.

How developers enable it

Developers can activate Tool Retrieval by including a tool_search tool in the request. They then mark selected tools with defer_loading set to true. The model still knows those tools exist, but their schemas are loaded only when required.

It can also connect to remote MCP servers in the Gemini API, which may expose tools to the model. Marking an mcp_server tool with defer_loading set to true makes all tools from that server load on demand. This allows a large server to be connected without naming every tool separately.

For teams that run their own registry, embedding index, a tool-finding structure, or user permission filter, Google provides a client-side option. They declare tool_search in client mode and define the search parameters. This method adds one extra round trip because the application performs the lookup. It also allows new tools to be exposed during a conversation, for example after a user connects an integration.

Caching, debugging and costs

Tool Retrieval is designed to work with prefix caching, a method that reuses the start of a prompt. Schemas that are deferred remain absent from the opening prompt, while loaded schemas appear later in the conversation. This can keep the cached opening stable across turns, which may help performance and cost.

Google says there is no separate charge for Tool Retrieval. Token charges for input and output follow the normal rates of the selected model. Enabling tool_search adds a small fixed amount of input tokens. Each schema the model loads is charged as input for the remaining conversation, like a tool declared at the beginning.

Savings depend on usage patterns. Google says savings occur when unloaded schemas outnumber loaded ones, which it describes as common with large libraries. Developers are advised to measure the feature against their own traffic.

Models and practical impact

Illustration: AI, Gemini logo, Google logo, settings and the labels Gemini 4, Gemini API, Google

The preview works with Gemini 3.6, later Gemini models and all Gemini 4 models in the Interactions API. Google says the feature is intended for complex agent tasks where many tools are available but only a few are needed for a given request.

For developers building AI agents, the feature gives more control over context size. Context size is the amount of information sent to the model. Instead of paying for and sending every tool schema each time, an application can load only what the current task requires. Google says this can reduce token use and delay while helping tool choice remain reliable as the library grows. Google has published documentation and a setup guide with examples.

Sources

  1. Introducing Tool Retrieval in the Gemini APIGoogle AI Blog

Tags

Spotted a mistake? See our corrections policy. Read how News by OnDevice works in our editorial policy.