Most companies send every AI task to the same heavyweight model. Classifying an invoice, pulling an address out of an email, categorising a ticket: it all goes to the same frontier model at the same rate. That is about as efficient as sending a lorry to deliver a letter.
A small language model (SLM) is a language model with roughly one to fifteen billion parameters, small enough to run on a single GPU or even a well-specified workstation, and therefore many times cheaper per unit processed than a frontier model behind an API. For narrow, repetitive tasks, the quality gap has closed far enough that the price gap no longer makes sense.
This guide covers what small models can and can't do, what they save in practice, and how model routing gets you the best of both.
What counts as a small language model?
There is no formal cutoff, but in practice the term covers roughly one to fifteen billion parameters. Familiar examples come from the Mistral, Llama, Qwen, Gemma and Phi families, nearly all with open weights you're allowed to run yourself.
The defining property isn't a score on a general benchmark; it's where the model fits. A model this size runs on one professional GPU, in a private cloud, or on a modern Apple Silicon machine. That removes the main reason to send your data to an external API.
What a small model is not: a slimmed-down frontier model with the same breadth of knowledge. It is a model that performs usefully on a narrower field. Judging one on general knowledge measures the wrong thing. For a broader comparison of available models, see our guide on open-source AI models.
Why is a small model often good enough?
Business tasks are rarely open questions. They repeat, they have a limited set of possible outcomes, and they stay inside one domain. That is exactly where small models perform well, particularly after tuning on examples from your own operation.
| Aspect | Small model (1-15B) | Frontier model via API |
|---|---|---|
| Cost per unit processed | very low, often infrastructure only | many times higher |
| Where your data sits | in your own environment | with the provider |
| Latency | milliseconds to a few seconds | depends on the service |
| Broad knowledge and reasoning | limited | strong |
| Complex multi-step tasks | weak | strong |
| Dependency on a vendor | none, with open weights | complete |
The shift of recent years sits in those first rows. Industry reporting puts the quality gap on narrow, domain-specific tasks far lower than it was, while the price gap has stayed wide. That changes the calculation for repetitive work compared with two years ago.
Treat those as indicative market ratios rather than a promise about your specific task. The only reliable test is your own test set of one hundred to three hundred real cases, measuring both options side by side.
What does it save?
The difference is structural, not just a lower rate. With an API you pay per unit processed, so your bill grows in direct proportion to success. With your own small model you pay for hardware or rental, and additional volume after that costs close to nothing.
That makes the crossover point a function of volume. At a few hundred operations a month an API is almost always cheaper and simpler. At tens of thousands a month it reverses, and at hundreds of thousands the difference stops being subtle. Our analysis of AI without API costs works that crossover through step by step.
There is a second saving that gets overlooked: with your own model the variable line disappears from your budget. You can roll a process out company-wide without costs moving with it, which makes the business case for automation far simpler.
[ TIME SAVED ]
Save 7 hours per week on repetitive classification of incoming documents and emails
Which tasks suit a small model?
The dividing line runs along the breadth of the task, not its difficulty.
Well suited to a small model:
- Classification and labelling (tickets, emails, documents, sentiment)
- Extracting structured data from free text (amounts, dates, names, references)
- Summarising short to medium texts within one domain
- Drafting standard replies from a fixed knowledge source
- Translating and rewriting within a fixed format
- Pre-processing: filtering, routing and enriching before a heavier model sees anything
Better on a frontier model:
- Multi-step reasoning where intermediate steps affect each other
- Tasks needing broad general knowledge outside your own domain
- Writing or understanding complex code
- Open-ended customer conversations where anything can come up
- Exceptions and edge cases where the question itself is unclear
The practical conclusion is that most companies need both, but rarely in the proportion they currently use.
How do you set up model routing?
Model routing means splitting your traffic: predictable tasks to a small model, exceptions to a large one. In modern production setups a large share of volume goes to the small model, with only the complex cases escalated.
A workable approach in four steps:
- Measure your current traffic. Which tasks go to an AI model today, how often, and what does each type cost? Without that breakdown, any saving is a guess.
- Pick one repetitive task with high volume and a limited set of outcomes. Classification is almost always the best first candidate.
- Build a test set and compare. Run the same one hundred to three hundred cases through both models and compare scores. Accept the small model only above an agreed threshold.
- Build an escalation rule. On low confidence, an unfamiliar pattern or an explicit exception, the case still goes to the large model or to a human.
That escalation rule is what makes the setup safe. You aren't choosing between cheap and good; you're choosing cheap where it works and good where it's needed.
What hardware do you need?
For a model in this class, video memory is the deciding factor rather than processor speed. A single professional GPU covers most business setups, and for smaller models at low volume a well-equipped Apple Silicon machine is enough for a proof of concept.
You don't have to buy immediately. Renting a private GPU from a European provider is a common intermediate step: your data stays in a defined environment without an upfront investment. Our guide on AI hardware for business covers buying versus renting with concrete guidance on memory per model size.
Whether that step makes sense at all depends on volume, data sensitivity and available skills. Our assessment in when to choose local AI gives you a decision framework.
Conclusion: choose the model per task, not per company
Small language models aren't a budget replacement for frontier models. They're the right tool for most of the work companies actually automate. Repetitive, well-defined tasks don't need a model that can converse about everything.
The gain is in the combination: route predictable volume to a small model you run yourself, escalate the exceptions, and measure both against the same test set. Our pillar on local AI for business sets out the wider picture, and our local AI work helps determine the crossover point and the setup.