Last checked: 7 November 2026
In short: A transformer is the design behind most modern AI language models. It reads a whole passage at once and works out which words relate to which, so it handles context far better than older designs.
Table of Contents
How it works
- Splitting. Text is broken into tokens, small pieces of words, and each one becomes a list of numbers.
- Attention. The model weighs how strongly each token relates to every other token in the passage.
- Prediction. Using those links, it predicts the most likely next token, then repeats the process.
Where you’ll meet it
ChatGPT, Gemini, Claude, Microsoft Copilot and Google Translate all run on transformers. The “T” in GPT stands for transformer.
What it means for your business
- It sits inside the tools you buy. You do not need to understand transformers to use them, but they explain why AI improved so much after 2017.
- Longer text costs more. Attention takes more work as text grows, which is why providers charge by length and set limits.
- The design is shared. Most AI firms use the same basic design, so the real differences come from data, tuning and price.
Watch out for
- No built-in fact check. A transformer predicts likely words, which is why it can produce fluent errors.
- Heavy computing needs. Large transformers need costly chips to train and run.
Example
Cohere. Aidan Gomez co-wrote “Attention Is All You Need”, the 2017 Google research paper that introduced the transformer. He later co-founded Cohere in Toronto, which builds transformer-based models for businesses. In July 2024 Cohere said it had raised $500 million at a $5.5 billion valuation, with investors including Cisco, AMD and Fujitsu. It has raised more since.
Transformer vs neural network
| Transformer | Neural network | |
|---|---|---|
| What it is | One specific neural network design | Software made of layers of connected units that learn from data |
| Main trick | Attention, linking every word to every other | Adjusting connections to reduce errors |
| How they relate | A type of neural network | The wider family |
Frequently asked questions
Why did transformers change AI?
They let models learn from huge amounts of text quickly, because they process words in parallel rather than one by one. That made today’s large language models possible.
Are transformers only used for text?
No. The same design now handles images, audio, video and even protein structures. Many image and speech tools use transformers alongside other methods.
Will something replace the transformer?
Possibly. Researchers are testing designs that handle long text more cheaply. For now, almost every leading AI model still has a transformer at its centre.