Also called: SLM · Last checked: 7 November 2026
In short: A small language model (SLM) is a language model compact enough to run on a laptop, phone or single server. It does less than a large model, but it costs less, answers faster and can keep data on your own device.
Table of Contents
How it works
- Fewer parameters. It has up to about 10 billion parameters (the numbers that store what it learned), against hundreds of billions in large models.
- Focused training. Developers train it on carefully chosen data, often with help from a larger model.
- Local use. It runs on the device itself, so it can work quickly and offline.
Where you’ll meet it
Apple Intelligence on iPhones, Gemini Nano on some Android phones and Microsoft’s Phi models use small language models for tasks on the device.
What it means for your business
- Lower running costs. An SLM costs a fraction of a large model per task, which adds up at high volume.
- Data stays in-house. Running a model on your own hardware keeps sensitive data off outside servers.
- Match the model to the job. Use SLMs for narrow, repeated tasks such as sorting emails or pulling details from forms.
Watch out for
- Less general knowledge. Small models make more mistakes on broad or complex questions.
- Setup effort. Running a model yourself needs technical skills and ongoing upkeep.
Example
Mistral AI. In October 2024 the French firm Mistral AI released Ministral 3B and Ministral 8B, two small models built for use on devices, such as offline assistants and on-device translation. Mistral said the 3B model beat its older Mistral 7B on most benchmarks. It priced the 3B version at $0.04 per million tokens through its own platform.
Small language model vs large language model
| Small language model | Large language model | |
|---|---|---|
| Size | Up to about 10 billion parameters | Hundreds of billions of parameters |
| Runs on | A laptop, phone or single server | Large cloud data centres |
| Best for | Narrow, repeated tasks | Broad, complex questions |
Frequently asked questions
Can a small language model work without the internet?
Yes. Once it is installed on a device, it can run offline. That suits field staff, secure sites and any task where data must not leave the building.
How small is a small language model?
There is no fixed cut-off. The term usually covers models with up to about 10 billion parameters, which can run on one good computer.
Is a small model good enough for customer service?
For simple, well-defined questions, often yes. For complex or unusual queries, pair it with a larger model or a person who can take over the conversation.