Direct answer
Agents degrade quietly rather than failing loudly, so support is mostly watching the right signals: refusal rate, rejection reasons, input freshness, and cost per run. Add provider version changes treated as releases, a pause path tested before it is needed, and named owners for approval, platform and drift.
AI agent maintenance
The ongoing work of keeping an agentic system healthy after release: monitoring behavioural signals, responding to provider and schema changes, curating evaluation cases from real rejections, and holding the ownership needed to pause or adjust it.
The routine, and why it is short
Most of this takes little time once it exists, and needs workflow understanding rather than machine learning expertise.
Read the refusals
A rising rate usually means an interface or the input data changed underneath.
Sample approvals in full
Approval rate alone hides reviewers who have started rubber-stamping.
Cluster rejection reasons
The cheapest source of new eval cases, because each is a real failure someone found.
Check input freshness
A stopped sync is silent, and the agent keeps producing confident output from stale data.
Watch cost per run
A quiet climb usually means retries or growing context.
Provider changes are releases, not events
A model version change is a behavioural change you did not make. Pin the version you validated, record which version each run used, re-run the eval suite before adopting a new one, and keep the previous version available to fall back to. Then a shift in behaviour is deliberate and evidenced.
Ownership and the pause path
Three duties need names before launch: a business approver, an engineering owner for the interfaces and providers, and someone with standing to pause the agent. Pausing should be a switch rather than a deployment, and it should be exercised once in advance so nobody discovers during an incident that the path was never tested.
The Propose–Decide–Execute pattern
CodeDTX builds agentic systems on the Propose–Decide–Execute pattern: agents may only write proposals with evidence attached, a named human records an approval or rejection with a reason, and a separate execution layer carries out approved work and logs the artifact. No agent holds a write tool to the outside world.
- 01
Propose
The agent analyses live system state and drafts a change, with the evidence it relied on attached to the proposal.
- 02
Decide
A named human approves, edits, or rejects with a reason. Risk tier determines who is allowed to decide.
- 03
Execute
A separate execution layer performs approved work — merge, publish, call, write — and records the resulting artifact.
- 04
Audit
Actor, reason, evidence, artifact, tokens, and cost are retained for every run, so any decision can be reconstructed later.
The six layers we build and review against
- 1
Agent layer
Agent architecture, tool use, memory and context, multi-agent patterns, structured outputs, orchestration.
- 2
Integration layer
MCP servers, tool contracts, API and database adapters, authentication, permissions, legacy system access.
- 3
Knowledge layer
Retrieval and RAG, vector and search architecture, enterprise knowledge sources, data access controls.
- 4
Reliability layer
Evals, tracing, observability, cost and latency budgets, fallbacks, regression tests.
- 5
Safety layer
Guardrails, prompt-injection defense, PII and data boundaries, human-in-the-loop gates, audit trails.
- 6
Product layer
The application people actually use: interfaces, approval queues, and operational runbooks.
Frequently asked
How much ongoing effort should we budget?
Allocated time inside existing roles rather than a dedicated headcount at the start. The common failure is ambiguity rather than under-resourcing — three people each assuming another is watching.
Can the build team hand this over?
Yes, and the handover should be part of delivery: the routine written down, the signals documented, and the business approver already approving rather than being introduced after launch.
What is the earliest sign an agent is degrading?
The refusal rate, followed by rejection reasons clustering on a new theme. Both move before anyone reports a problem, which is the point of watching them.
Have a workflow that should become AI-enabled?
Tell us about the system it lives in. We reply from an engineering seat, not a sales deck.