If a rule isn’t code, it’s a wish
An autonomous system almost committed the business to a specialist role that did not exist. The gates were written down. They were not enforced.
Rule: If a rule isn’t code, it’s a wish.
The near-miss
In a real production pipeline — opportunity in, deliverable out — an agent assembled a package that implied a capability the team did not have. Humans would have caught it in a careful read. The machine did not, because nothing in the execution path forbade it. The prohibition lived in notes, prompts, and “we always check that.”
Prose is not a gate. Prose is a hope with formatting.
Why prompts fail as safety
Prompts are instructions to a statistical machine. They work often. They fail silently. They are rewritten by the next context window. They are not auditable the way a hard stop is auditable.
If a rule matters — money, reputation, legal scope, capability you do not have — it must fail closed in code, workflow, or a human signature before anything leaves the building.
What “code” means here
Not only software. “Code” means anything that executes without relying on the model to remember:
- a validator that rejects forbidden claims
- a checklist that blocks send until fields are proven
- a human-in-the-loop step that cannot be skipped by the agent
- a schema that will not serialize an invalid job
If the agent can talk its way past the rule, the rule is not real.
Transfer to your business
List every “we always…” about your AI or automation. For each: is it in the path, or in a doc? Move the three that would embarrass you most into hard stops this week.