Skip to main content

Gemini API Tool Retrieval: which tool schemas to defer

Google has added Tool Retrieval to the Gemini API in preview, so agents load a tool's parameter schema only when a task needs it. What changed, how server-side and client-side search differ, what it costs, and which tools to keep loaded.

Share -
Man in a dark suit pressing a glowing cyan On button on a holographic interface surrounded by hexagonal tool icons in a modern office

Gemini API Tool Retrieval, now in preview, lets an agent see only the names and descriptions of most of its tools and fetch a tool's full parameter schema when a task calls for it. For teams with large tool libraries, the decision is which tools to defer, and whether Gemini or your own registry should do the search.

What Google released

Tool Retrieval was announced by Google on 19 November 2026 as a preview feature of the Gemini API's Interactions API. It works with Gemini 3.6 and later, including every Gemini 4 model. The setup guide is in the Google AI Studio documentation.

Until now, an agent on the Gemini API sent every tool's full parameter schema with every request. Google's announcement names three costs of that as the tool list grows: more input tokens, more latency, and less accurate tool selection.

How Tool Retrieval works

There are two modes.

Server-side search. You add a tool_search entry to the request's tools and set defer_loading: true on each function you want to hold back. The model sees a deferred tool's name and description but not its parameters. When it needs one, it searches by name, the API loads the schema, and the model emits the function call, all inside a single interactions.create request. The interaction's steps record this as tool_search_call, tool_search_result and function_call, so the lookup is visible in your traces. Google runs the search, so there is no index for you to build or host.

Client-side search. You declare tool_search with "execution": "client" and your own parameters, such as a query string. When the model calls it, your application returns matching function schemas in a function_result. Google notes this adds a round trip per search, but it means tools don't have to be declared up front and new ones can be exposed mid-conversation.

Two further details matter for agent teams. Setting defer_loading: true on an mcp_server tool makes every tool on that remote MCP server load on demand. And because deferred schemas stay out of the start of the prompt, prefix caching keeps working across turns.

What it costs

According to Google's announcement, there is no separate charge. Turning on tool_search adds a small fixed block of input tokens, and every schema the model loads is billed as input for the rest of the conversation at the model's standard rate. Google expects savings when the schemas left unloaded outweigh the ones loaded, and advises measuring this on your own traffic. It publishes no savings figures, so treat any number you have seen elsewhere with caution.

Which tools to defer

This is CodeDTX's advice for teams running agents with more than a handful of tools.

  • Keep the core tools loaded. Google's guidance is to leave frequently used tools undeferred and defer the long tail. In practice that means the tools an agent calls on most tasks: lookup, search over your own data, the main write action. Deferring them only adds a search step to almost every request.
  • Defer the long tail by domain. Rarely used admin, reporting and back-office tools are the natural candidates. Their descriptions now carry more weight, because the model chooses on name and description alone. Write them as if they were the whole contract; our guide to designing tools an AI agent can use covers what a good description includes.
  • Defer whole MCP servers carefully. One flag on an mcp_server tool defers everything it exposes, which is convenient for large third-party servers. Check that each tool's description on that server is clear enough for the model to find it. See when an agent needs MCP or A2A for where MCP fits.
  • Use client-side search when permissions matter. If the tools a user may call depend on their role, tenant or approval state, return schemas from your own registry so a tool the user cannot use is never offered. The extra round trip is the price of keeping that rule in your code rather than in a prompt.

Migration and governance checks

Tool Retrieval is only in the Interactions API, so an agent built on another Gemini API surface has to move to it first. Google's announcement says nothing about Vertex AI, so teams on Google Cloud should check the Vertex AI documentation before planning around it.

Before switching, record a baseline: input tokens per task, latency and how often the agent picks the wrong tool. Re-run the same evaluation set with tools deferred. Watch for tasks where the model never finds the right tool because its description is vague; these show up as wrong or missing calls rather than errors, so our notes on handling a failed tool call apply. Make sure your traces keep the tool_search_call and tool_search_result steps, because an auditor will want to see which tools the agent looked at, not only the one it called. For cost tracking, see how to control AI agent running costs.

When not to switch yet: if your agent has a small, stable tool set, the fixed tool_search overhead and the search step may cost more than they save. And a preview feature can change, so keep the existing request path available for production until it reaches general availability.

Frequently asked questions

What is Tool Retrieval in the Gemini API?

Tool Retrieval is a preview feature of the Gemini Interactions API that lets an agent load tool parameter schemas on demand. You add a tool_search entry and mark tools with defer_loading, so the model sees only their names and descriptions until it needs one. Google announced it on 19 November 2026 for Gemini 3.6 and later models, including the Gemini 4 family.

Does Tool Retrieval cost extra?

Google says there is no separate charge. Enabling tool_search adds a small fixed block of input tokens, and each schema the model loads is billed as input for the rest of the conversation at the model's normal rate. Savings depend on how many schemas stay unloaded, so measure input tokens per task on your own traffic before and after turning it on.

Use server-side search when your tool list is fixed and you want no extra infrastructure, because Gemini runs the search inside the same request. Use client-side search when you already have a tool registry, need to filter tools by user permissions or tenant, or add tools during a conversation. Client-side search costs an extra round trip per search, which adds latency.

Does deferring tools break prompt caching?

No. Google states that deferred schemas are kept out of the start of the prompt, so prefix caching keeps working across turns. Schemas that the model loads during a conversation are added later and billed as input from then on. Keep the set of always-loaded tools stable between requests so that the cached prefix stays the same and can be reused.

Which Gemini models support Tool Retrieval?

According to Google's announcement, Tool Retrieval works with Gemini 3.6 and later models, including all Gemini 4 models, when called through the Interactions API. The announcement does not mention Vertex AI or a date for general availability, so treat it as a preview feature and check the Google AI Studio documentation for changes before relying on it in production.

Share this post

Contact us to build the right product

Talk to our engineers about your application, the systems it connects to, and what you want to build next.

Get in touch
Two people discussing work with a laptop