Most calculations around local AI stop at the purchase order. Two GPUs, a server, an implementation project, done. That is the cheap part.
Running your own AI environment costs roughly €15,000 to €40,000 per production server per year from year two onward, and that figure is dominated by people rather than by hardware or electricity. Budget only for the purchase and you have budgeted about a third of the real bill.
What follows is a worked model with the assumptions stated, so you can rerun the arithmetic with your own numbers. The amounts are indicative and meant as a calculation framework, not as a quote.
What sits inside the running cost?
Running a live AI environment breaks into five cost lines, and they differ sharply in how predictable they are. Power and colocation can be budgeted almost exactly. Model upgrades and incidents cannot.
| Cost line | Indicative per year | Predictability |
|---|---|---|
| Power and cooling | €1,500 to €3,300 | High |
| Colocation or floor space | €2,000 to €6,000 | High |
| Hardware depreciation | €9,000 to €12,500 | High |
| Model upgrades and retesting | €3,000 to €8,000 | Medium |
| Maintenance, patching, on-call | €9,000 to €18,000 | Low |
The bottom two lines are where budgets break. Neither appears on any quote, because neither is a purchase. They are internal time.
What does the electricity actually cost?
Power is the line everyone overestimates. Work it out once and you can stop worrying about it.
An NVIDIA L40S, the card you meet most often in business inference, draws a maximum of 350 watts and carries 48 GB of memory. A server holding two of them, plus host, memory and storage, pulls around 1.1 kW under full load. In practice a machine like that rarely runs at peak around the clock: with daytime business use the average sits closer to 0.5 kW.
Continuously at peak, that is 9,636 kWh per year. At a more realistic 0.5 kW average you land at 4,380 kWh. Dutch statistics office CBS recorded €0.241 per kWh for business consumers in the 500 to 2,000 MWh bracket in the second half of 2025, taxes included. Add forty percent for cooling and you get €1,478 to €3,251 per year.
That is less than a single month of API spend on a mid-sized deployment. Electricity is not an argument against running your own. The argument lives somewhere else.
How often do you have to replace the model?
This is where the real work starts. A local model is not an appliance you install and forget: it ages at the speed the market ships new models, and right now that is a meaningful jump in quality per parameter every six to nine months.
Each replacement is a small project. You pull the new model, run your own evaluation set through it, compare the output against the previous version, adjust prompts and system instructions wherever behaviour has shifted, and roll out. For an environment carrying two or three applications, that cycle costs two to four days of work.
Two upgrades a year at three days each, charged at an internal rate of €600 per day, comes to €3,600. Skip it and in eighteen months your organisation is running on a model noticeably worse than what your competitor pulls from an API. Skipping this line is the most common reason a private environment gets written off as a failure after two years.
Starting smaller lowers the bill here. Small language models are faster to retest and less demanding of the hardware underneath them.
[ TIME SAVED ]
Save 6 hours per week on manually retesting prompts after every model upgrade
Who carries the pager?
The most expensive line, and the worst estimated. An AI environment in production carries the same operational duties as any other business system: security patches on the operating system and the inference stack, monitoring on memory use and response times, backups of configuration and vector indexes, and somebody reachable when it stops responding at half past seven in the morning.
Budget 0.1 to 0.2 FTE for a single production environment with a few applications on it. At fully loaded employer costs around €90,000, that is €9,000 to €18,000 a year. The number barely moves when you go from one server to two, which is exactly why running your own scales: the fixed maintenance load is the threshold, not the marginal cost.
Three ways to cover that line, ordered by how often they work:
Give it to your existing IT provider as an extension of the server work they already do. This works when the environment sits in your own data centre or colocation and there is nothing exotic about it.
Buy maintenance from whoever built the environment, at a fixed monthly fee with agreed response times and model upgrades included. More expensive per hour, but knowledge of your specific setup comes with it.
Hand it to a systems administrator internally as an extra duty. Cheap on paper. It holds while that person is there and falls over the day they leave. Choose this route and you must keep the setup documented and reproducible, or you have bought dependency on one employee instead of dependency on one vendor.
When is running your own not the cheaper option?
Most comparisons dodge this question, so here is the answer.
Add up your API spend over the past six months and divide by six. If that monthly figure sits below €1,500, a private environment probably will not pay for itself on financial grounds alone. You would be setting €18,000 of annual API cost against €15,000 to €40,000 of maintenance, buying work in rather than out. Other reasons may still decide it, and often they do: data that cannot leave the building, a sector with its own regulator, or a client contract that mandates processing inside the EU. Call that a sovereignty decision, though, not a saving.
Above €4,000 a month the picture tilts. There you set €48,000 of annual API cost against a maintenance load that is largely fixed, and that you spread across several applications. At that scale the question stops being whether to run it yourself and becomes when.
In between it is a strategic call. The full comparison of both models is in our guide to local AI for business, and the build sequence itself in implementing an on-premise LLM. If you want to cost the hardware before anything is ordered, the configurations are in AI hardware for business.
What your budget has to contain
A budget carrying these five lines survives year two. A budget listing only the purchase becomes an argument in year two.
Put hardware depreciation in as a fixed annual line over four years, not as a one-off. Enter power and cooling with your own kWh rate and a realistic load assumption rather than the number on the spec sheet. Budget two model upgrades a year at a fixed number of days each. Record the maintenance FTE as an amount with a name under it, internal or external. And reserve a line for the first incident, because it is coming.
An environment budgeted this way stops being an experiment and becomes property: a system on your own balance sheet, with costs that do not move with your usage, and a supplier you can replace without moving your data. That is the case for running your own. The power bill is not.
If you would rather not have the environment in your building, the middle path is worth weighing: the same control, hosted with a European provider. What to select one on is covered in choosing a sovereign cloud.