I Asked My AI If We Were Overdoing It. Twice.

I am building a factory. Not a building with machines in it: a system of AI agents whose job is to build other AI agents.

The first agents it builds are for the pharmaceutical industry. They answer questions about a medicine, using only the company’s approved documents and nothing else. In that world a wrong answer is not an inconvenience. It is a compliance problem.

So every answer gets marked. A separate AI acts as the examiner. It reads each answer and checks it, line by line, against the approved documents and against a growing book of rulings on what counts as a pass and what counts as a fail.

This week we ran our biggest test so far. Thousands of answers, every one of them marked.

The question I asked twice

I asked my AI assistant whether we were overdoing it. Then I asked again. Both times the answer came back calm and confident: no, this is fine.

Neither answer had a number behind it. Nobody had measured what marking a single answer actually cost.

When the run finished, close to half of my weekly AI allowance was gone.

Where it went

Writing the answers was the small part. The marking was the bill.

The reason was simple once we looked. Before marking each answer, the examiner re-read its entire marking pack: hundreds of pages of rules, past rulings and reference material. Every answer, every time.

Picture a teacher who re-reads the whole textbook before marking each exam paper. Every paper gets marked correctly. The term is over before the marking is.

The waste was not the real lesson

The waste is fixable. The examiner will get only what each marking needs. It will mark several answers at a time. And before it marks anything real, the leaner examiner has to prove it reaches exactly the same verdicts as the old one. Quality is not traded for savings.

The real lesson is the two answers I got. They sounded certain. They were not measured. From the outside, an AI that sounds sure and an AI that has checked look exactly the same.

Confident is not the same as measured. From the outside, they look identical.

So there is a new rule. When I ask about cost or scale, the answer is a measured number, or “we haven’t measured it, measuring now.” And every large run states what it will use before it starts, so I see it first.

A question for your own AI

Next time your AI tells you something with confidence, ask it one more thing: what is that based on? If the answer isn’t something you can check, the confidence doesn’t count yet.

That is much of what we do at AI.me: AI whose answers can be checked, kept that way after launch. You drive. We handle the maintenance.

Dennis Noel Gomez is the founder of AI.me, an applied AI solutions provider in Riyadh, Saudi Arabia. He builds working executives an AI second brain and runs one himself. These notes are written from the builds as they happen.

Part one of a series on the lessons from building an AI factory, the costly ones first.