Skip to main content

Tool schema drift: keep AI agents working when APIs change

When an API behind an agent tool renames a field or tightens a rule, the agent keeps calling the old shape. Pin tool contracts, test them in CI and watch for drift.

Share -
A woman with curly hair types at a desk in a brick-walled office while a monitor beside her shows a glowing AI brain wireframe surrounded by blue system diagrams.

Tool schema drift happens when the API behind an AI agent's tool changes shape, but the agent still calls it the old way. A renamed field, a new required parameter or a stricter enum breaks the agent quietly. Treat every tool as a contract: pin its schema, test it on every change and alert when the live API stops matching.

Why schema drift hits AI agents harder

A normal integration fails loudly when an API changes: the code no longer compiles, or a test goes red. An AI agent is different. It reads a tool description and a JSON schema, then builds the call itself at run time. If the schema it was given no longer matches the API, the model will still produce a call that looks plausible.

The result is rarely a clean error. The API may ignore an unknown field, fill a default the agent never meant to choose, or return a partial result that the model then summarises as if it were complete. Users see an answer that sounds right and is not.

Drift comes from several directions at once:

  • Third-party APIs add required fields, deprecate parameters or change validation without much notice.
  • Internal services evolve on their own release schedule, owned by teams who may not know an agent depends on them.
  • MCP servers can change their tool list or input schemas between sessions. The Model Context Protocol tools specification lets a server announce this with a notifications/tools/list_changed message, but only if the client listens and refreshes.

Where drift breaks an agent

Calls that validate but mean something else

The most damaging case is a call that still succeeds. A field renamed from amount to amount_minor might be dropped silently, so a refund goes through with a default value. Nothing fails, and the audit trail shows a successful action.

Stale tool descriptions

Even when the parameters still match, the description can drift from the behaviour. If an endpoint now returns only active records, but the description still says "all records", the agent will draw the wrong conclusion from an empty list.

Cached tool lists

Agent clients often cache the tool list they fetched at start-up. If the server changes underneath, the agent keeps offering the model a tool that no longer exists, or misses one that does. That turns into a run of failed tool calls that the agent may retry without ever recovering.

How to keep tool contracts in step

Own the tool schema as a versioned file

Keep each tool's name, description, input schema and output schema in source control, next to the agent code. Do not generate them on the fly from whatever the API returns today. A change to the contract should be a reviewed change, the same as a change to a prompt, and it should travel with the agent version so you can roll back both together.

Put an adapter between the agent and the API

Give the agent a tool shaped for the task, not a raw copy of the vendor's endpoint. A thin adapter maps the stable tool contract onto the current API. When the API changes, you update the adapter and its tests, and the schema the model sees stays the same. This also keeps tools designed for how an agent reasons rather than for how a vendor organises its endpoints.

Test the contract in CI, against the real thing

Run contract tests on every pull request and on a schedule against a sandbox of the real API:

  • Send a known valid call and check the response still matches the output schema the agent expects.
  • Send a call missing each required field and confirm the API rejects it, so you know your schema's required list is still accurate.
  • Diff the live tool list from each MCP server against the pinned copy and fail the build on any unreviewed change.

Scheduled runs matter because third-party APIs change when their owners decide, not when you deploy.

Validate at run time as well

Check every call the model builds against the pinned schema before sending it, and check every response before handing it back. Reject unknown fields instead of letting them pass through. That turns silent drift into a clear, logged error the agent can report, which is the same discipline as validating agent output before it reaches your systems.

Refresh tool lists deliberately

If your agent uses MCP, subscribe to list change notifications and re-fetch the tool list when one arrives. Then compare the new list with the pinned version. A changed schema should pause that tool and alert the owner, not flow straight into the next prompt.

Who owns the contract

Drift is often an ownership problem. The team running the agent assumes the API team will warn them, and the API team does not know the agent exists. Record every tool in your agent inventory with the API it calls and the team that owns that API, and add the agent team to that API's change notices. Put drift alerts on the same dashboard you use to monitor the agent in production.

If you are connecting agents to systems you do not control and want a second view on how to keep those integrations stable, talk to our team.

Frequently asked questions

What is tool schema drift in an AI agent?

Tool schema drift is a mismatch between the schema an AI agent uses to call a tool and the way the underlying API now behaves. It happens when a field is renamed, a parameter becomes required, an enum changes or a response gains new structure. Because the model builds calls from the schema it was given, it keeps sending the old shape, which can fail or, worse, succeed with the wrong meaning.

How do you detect schema drift before users notice?

Run contract tests on every pull request and on a regular schedule against a sandbox of the real API. Compare responses with the pinned output schema, confirm required fields are still enforced and diff each MCP server's live tool list against the copy in source control. Add run-time validation of every call and response, and alert on rejected calls so drift shows up as a clear error rather than a confusing answer.

Does MCP stop tool schema drift?

Not on its own. The Model Context Protocol lets a server send a list changed notification when its tools change, and newer versions support output schemas that results must match. That only helps if your client listens, re-fetches the tool list and compares it with what you expect. Treat a notification as a trigger to check the change and pause the affected tool, not as permission to use the new schema straight away.

Should an AI agent call a vendor API directly or through an adapter?

An adapter is usually safer. It gives the agent a stable tool shaped around the task, while the adapter handles the vendor's current field names, versions and validation rules. When the vendor changes the API, you update and test the adapter, and the schema the model sees does not move. Calling the vendor API directly ties the agent's behaviour to every change the vendor makes, often without warning.

Share this post

Contact us to build the right product

Talk to our engineers about your application, the systems it connects to, and what you want to build next.

Get in touch
Two people discussing work with a laptop