Also called: XAI, explainability · Last checked: 7 November 2026
In short: Explainable AI (XAI) is AI designed so people can understand why it reached a decision, such as which factors led to a loan being refused. It lets staff, customers and regulators check and challenge results.
Table of Contents
How it works
- Choose a suitable model. Some models, such as simple scoring rules, are easy to explain. Complex ones need extra tools.
- Measure each factor’s influence. Explanation tools estimate how much each input, such as income or missed payments, affected a result.
- Turn it into plain words. The system gives reasons a person can read, such as “your credit history is too short”.
Where you’ll meet it
Credit refusals, insurance quotes and fraud alerts increasingly come with reasons. ChatGPT and Claude can show the sources behind an answer, a related step towards explaining output.
What it means for your business
- Customers expect reasons. People refused credit, a job or a claim want to know why, and data protection law can give them that right.
- Choose explainable tools for big decisions. For hiring, lending or pricing, ask suppliers how the tool explains each result before buying.
- Train front-line staff. Service teams need simple explanations they can pass on when customers complain.
Watch out for
- Explanations that mislead. Some tools give a tidy reason that does not match how the model really decided.
- Accuracy trade-offs. The most accurate model is sometimes the hardest to explain. Decide which matters more for each use.
Example
Goldman Sachs. In November 2019 customers complained that the Apple Card, issued by Goldman Sachs, gave men far higher credit limits than their wives. In March 2021 the New York Department of Financial Services found no unlawful discrimination and said the decisions could be explained. It also found that poor customer service and a perceived lack of transparency had undermined trust.
Explainable AI vs AI transparency
| Explainable AI | AI transparency | |
|---|---|---|
| Question | Why did the AI decide this | What AI is used, and how |
| Level | One decision or result | The whole system and its use |
| Example | Reasons for a refused loan | Telling customers a chatbot is AI |
Frequently asked questions
Do all AI systems need to be explainable?
No. A tool that suggests email subject lines needs little explanation. Systems that affect people’s money, jobs or health need clear reasons that staff can check.
Can large language models explain their answers?
Only partly. They can write a reason, but it may not reflect how they produced the answer. Asking for sources you can check is more reliable.
How do you test whether an AI explanation is any good?
Give the explanation to the person affected and to a staff member. If both can understand it and act on it, such as by correcting wrong data, it works.