Someone named, before launch. The duties split three ways: a business owner who approves what the agent proposes, an engineering owner who keeps the interfaces and providers it depends on healthy, and a decision owner who judges when behaviour has drifted enough to pause it. Often two people, rarely none.
Three duties that are usually confused
The approval duty belongs to the business. Whoever reviewed this work before the agent existed is normally the right approver, because they already hold the judgement and the accountability. Moving that to an engineer looks efficient and quietly relocates responsibility for business outcomes to someone without the context.
The platform duty belongs to engineering. Tool contracts break when a downstream schema shifts. Providers change model versions and deprecate old ones. Credentials expire. None of this announces itself in the agent's output; it shows up as degraded proposals or a rising refusal rate.
The drift duty is the one most often unassigned. Someone has to look at whether the agent is getting worse, and have the authority to pause it. Without that authority the practical response to a bad week is an argument, because pausing a system that leadership has been shown is a decision nobody wants to own informally.
What the operating routine actually contains
Most of it is unglamorous and takes little time once it exists.
- Read the refusals. A rising refusal rate is the earliest useful signal, and it usually means an interface changed or the inputs did.
- Sample the approvals. Read a handful of approved proposals in full. Approval rates alone hide the case where reviewers have started rubber-stamping.
- Watch the rejection reasons. These are the free evaluation set. Reasons clustering on one theme tell you what to fix next.
- Check freshness. If the agent reads data that syncs on a schedule, a stopped sync is silent and the agent keeps producing confident output from stale inputs.
- Track cost per run. A quiet climb usually means retries or growing context, both of which are worth understanding early.
Notice that none of this requires machine learning expertise. It requires someone who understands the workflow and has time allocated to it.
Naming it before launch, not after
The reason to settle this before launch is that the alternative is settling it during an incident. A pilot has an engineer watching closely; production has whoever remembers. When behaviour degrades, the failure is usually not that nobody noticed but that nobody had standing to act.
Write it down: the business approver, the engineering owner, who may pause the agent, and how a pause happens mechanically. That last part matters. If pausing means a deployment, it will not happen quickly, so a switch that stops the agent proposing is worth building before it is needed.
This connects directly to monitoring. How you monitor an agent in production covers what to instrument, and what happens when a model is deprecated covers the provider change that most often arrives unannounced. If you want the operating model designed alongside the build, that is what a dedicated AI engineering team is for.
Frequently asked questions
Can the team that built the agent operate it?
For a while, and it works well early because they recognise unusual behaviour quickly. It becomes a problem when the build team moves on and the operating knowledge leaves with them. Plan the handover as part of delivery: the routine written down, the signals documented, and the approver already doing the approving rather than being introduced later.
Does this need a dedicated person?
Rarely a full role at the start. It needs allocated time inside existing roles, and an explicit name against each duty. The failure mode is not under-resourcing so much as ambiguity, where three people each assume another is watching. One named owner with a modest time allocation beats a committee with none.
What does pausing an agent actually mean?
Ideally it stops the agent proposing while leaving the rest of the workflow intact, so people revert to the process they used before. That should be a switch rather than a deployment, and it should be exercised once before launch so nobody discovers during an incident that the pause path was never tested.
