Articles · applied AI from Saudi Arabia

I Wrote the Rule. It Broke It Anyway.

Four of the five mistakes came back, and each one went against a rule that was written down.

Dennis Noel Gomez17 November 20262 min read

A written rule card, RULE: WRITTEN, beside the number 4 of 5 mistakes returned

I build AI agents for regulated work, and I run an AI assistant for my own business. In early October I asked that assistant a question. Can another AI be trusted to catch the mistakes where an AI simply did not check something? It said yes. A rule such as “read the source first” would catch them.

I said no. A rule in a prompt is only text. The AI reads it, and sometimes it does not follow it. Only code that runs every time is certain.

I had a way to find out. I keep 15 test conversations for my assistant. Each one is built to tempt it into a known mistake, and I run them in fresh chats. On 19 October they found five mistakes. On 12 November I ran the same set again. Four of the five were back.

All four went against rules that are written in its instructions. On 19 October it described one of those rules back to me correctly: a view that was not the assistant’s must not be recorded as its own. Then it offered to record my view as its own.

Another time it described the correct next step, then asked me for permission to take it. Its instructions say to just do it.

Found on 19 Oct5 mistakesBack on 12 Nov4 came back
Mistakes found by the same 15 test conversations, 24 days apart.

That is why a person reviews every cycle of the agent factory I am building. A good rule lowers the number of mistakes. It does not take it to zero.

A written rule is a request. Only code that always runs is certain.

Get started

Built in Riyadh. Built around you.

A short conversation is usually enough to know whether this is for you.