The demo worked. The pilot worked. Three months later the same agent behaves differently than it did in week one, and nobody can say exactly when that started. This is the most common pattern with AI in companies, and it rarely has anything to do with the model.
Running AI agents in production is the practice of putting fixed measurement points, version control and ownership around a system whose output varies by design. With ordinary software a release is an event you schedule; with AI the underlying component changes without you doing anything. Without measurement, you notice that change when a customer complains.
This guide covers what to measure, how to keep quality stable through model changes, what running costs monthly, and who owns it.
Why do AI projects stall between pilot and production?
A pilot proves something can work under favourable conditions: selected cases, engaged staff, a short timeline. Production is the opposite: every case including the strange ones, handled by people who didn't choose the system, over years.
Three causes explain most failures. Edge cases weren't included, because the pilot deliberately used representative examples. No owner was appointed, so when quality slips it's nobody's job to notice. And there is no baseline, which leaves "it works less well than it used to" as a feeling rather than a number.
That last one is the easiest to fix and pays back the most. Set a test set before go-live and you still have a yardstick a year later. How to set up that first phase is covered in our guide on starting an AI pilot project.
What do you measure on an agent in production?
Four categories are enough. More measurement points rarely lead to better decisions; fewer definitely lead to blind spots.
| What you measure | Concrete metric | Why it matters |
|---|---|---|
| Quality | score on a fixed test set of 100 to 300 cases | detects silent degradation |
| Coverage | share of cases handled without a human | shows whether the time saving is real |
| Escalation | share going to a human, and why | points to the next improvement |
| Cost | cost per case handled | makes scaling predictable |
The most important of the four is cost per case, because that number carries your business case. An agent handling 80 percent of requests at twenty cents each is a different asset from the same agent at two euros each, even when the output is identical.
Always record the reason for escalation, not just the count. Escalations are the cheapest source of improvement information you have: they show exactly where the system hits its limit. For the wider set of steering metrics, see our guide on measuring AI results with KPIs.
[ TIME SAVED ]
Save 6 hours per week on manually sampling and correcting AI output without a fixed measurement
How do you keep quality stable through model changes?
This is where AI operations differ from ordinary application management. If you use a cloud model, the provider can replace or retire a version at a moment that doesn't suit you. Your system changes without a release.
Three measures cover it.
Pin your model version where the provider allows it, and treat a version change like any other change: announce, test, roll out. Automatically riding "the latest version" is comfortable until the day it isn't.
Run your test set on every change. Not only on a model swap, but also on edits to prompts, tools or the knowledge base. A prompt tweak that fixes one problem regularly creates two elsewhere.
Keep a rollback path. Knowing which version worked and being able to return to it within an hour is worth more than any attempt to fix the fault in place.
With self-hosted models the responsibility inverts: nothing changes without you, but security updates and model upgrades are now your job. That is exactly the trade-off in our comparison of local AI and cloud AI.
What does running an agent cost per month?
The build is one-off, running it is monthly, and that second number is usually underestimated at the start. Budget four lines.
Consumption or infrastructure. With a cloud model you pay per unit processed, and that bill grows with usage. With your own environment you pay for hardware or rental regardless of volume.
Operations and oversight. Someone reviews the metrics, assesses escalations and adjusts. For one productive agent that is a few hours a month in practice, not zero.
Knowledge source maintenance. Documents go stale. An agent answering from outdated procedures is accurate and wrong at the same time. Budget for periodic cleanup.
Continued development. Every month produces escalations that call for an adjustment. Without budget for that, the agent freezes at the level it had on delivery day.
Skip these four in your budget and the business case evaporates in year two. The ratio varies, but the rule of thumb is that annual running and development costs represent a substantial share of the original build cost.
Who runs the agent, and what does that look like?
Ownership is the highest-return, lowest-cost control there is. Appoint one person accountable for the agent's performance, preferably from the team that runs the process rather than from IT alone.
A workable rhythm for an agent in production looks like this:
- Weekly: review metrics, go through that week's escalations, flag notable cases.
- Monthly: run the test set, compare cost per case with last month, ship one improvement.
- On every change: run the test set before and after, and record the result.
- Quarterly: permissions review and a check that the agent still operates within its original remit.
That last item belongs with the controls in our guide on securing AI agents against prompt injection, because permissions that get widened over time are one of the most common causes of incidents.
Only once this rhythm holds for one agent does extending to more processes make sense. Our guide on scaling AI across the business covers that step.
What do you settle before an agent goes live?
A go-live decision runs more smoothly when the criteria are on paper beforehand. Six points cover most implementations.
The test set and the threshold. Which cases are in it, what score is the minimum, and who judges the outcome.
The scope. Which cases the agent may handle alone and which go to a human by definition, regardless of how confident the system is.
The fallback rule. What happens on uncertainty, during an outage, and on a pattern the agent doesn't recognise.
The owner and the rhythm. Who looks at which numbers when, and who they report to.
The running budget. Consumption, oversight, knowledge source maintenance and continued development, expressed per month rather than as a one-off line.
The stop criterion. At what result do we switch the agent off or roll it back to the previous version.
That last point is nearly always skipped and is the most valuable of the six. An agreed stop criterion takes the emotion out of a rollout that disappoints: you don't have to debate whether it's good enough, because that question was answered while nothing was at stake.
Equally useful is writing down what the agent explicitly does not do. An expressly bounded remit is easier to assess, and that boundary later becomes the starting point for expansion rather than an argument about what was originally intended.
Conclusion: measurement makes the difference, not the model
An AI agent in production isn't a delivered project but a running system with variable output. Set a fixed test set, four metrics, version control and one owner, and you keep control. Skip it, and you'll discover problems through customers.
The investment is modest: a day to build the test set and a few hours a month to steer. We build AI agents with metrics and logging from day one, because an agent you can measure is an agent you can improve.