Articles · applied AI from Saudi Arabia
The Phone Number That Was One Digit Off
Fifty passages in my AI's documents agreed on a phone number. One did not. The AI had no way to tell which was right.
One of my AI's source documents had a phone number that was one digit wrong. Fifty other passages had it right.
I am building an AI that answers questions about a medicine, using only the pharmaceutical company's own documents. Before I tested the AI at all, I had a second AI read those documents and check one simple thing: do they agree with each other?
They did not. It found six places where two documents said different things.
What it looked like
The phone number was the cleanest case. Fifty passages gave one number. A single passage gave another, one digit apart. Ask the AI for the number and it could have given either, in the same confident voice. Elsewhere, one document said the second pack was collected at week two and another said week four. A third gave a price in two forms that did not match.
None of this was the AI's fault. It reads what it is given. But a wrong phone number about a medicine, said with full confidence, is not a small mistake.
An AI is only as consistent as the documents it reads.
What changed
Where the official label settles a disagreement, the label wins. Where nothing settles it, the AI no longer picks a side. It says it cannot confirm and hands over the company's contact. The company's team reviews every disagreement before go-live, and the factory has one firm rule: no AI goes live on documents with an open disagreement.
The same goes for any AI that answers from your documents. Check the documents first.
Part seven of a series on the lessons from building an AI factory, the costly ones first.