In my last article, an agent reported a pull request that never existed.
The lesson was straightforward: an agent's output is a claim. System state is evidence.
So I moved more control into the pipeline.
This time, nothing lied. The plan was coherent, the reasoning defensible, the implementation direction technically sound.
It was still wrong.
The setup
My lab runs a gated agentic delivery pipeline on local models. A human defines and approves the work. A planning agent breaks it into contracted subtasks. Execution agents build against those contracts. Controls validate the work, and a human merges at the end.
The task was simple: give the pipeline access to secrets it needed inside my Kubernetes cluster.
The cluster runs Flux. That was documented in an ADR, so the planning agent had it in context.
The cluster also uses Sealed Secrets for secrets stored in Git. That decision wasn't documented yet.
The wrong namespace
I was trying to close two gaps at once. I needed the implementation done, and I needed the missing decision recorded. So I had the AI draft the ADR while it worked through the implementation.
The plan came back using SOPS.
That wasn't random. SOPS is a perfectly reasonable way to encrypt secrets in Git, and it's commonly used with Flux. The model had Flux in context. It didn't have Sealed Secrets. So it resolved the missing detail to the strongest answer available.
The result was confident, well structured, technically defensible — and not what I intended.
I led the witness
This wasn't a hallucination. The agent didn't invent a tool call, claim work it hadn't done, or pick a bad technology.
It answered an architectural question I had failed to answer for it.
If you write C#, you've seen a version of this. You mean one Timer, but the wrong namespace is imported. The name resolves. The code looks reasonable. It may even compile. It's just not the Timer you meant.
My ADRs are my imports. Flux was imported. Sealed Secrets wasn't. SOPS was the nearest valid symbol.
The difference is what happens next. A compiler can stop when a reference is ambiguous. A model generally won't. It resolves, and because the answer is fluent and internally consistent, the resolution looks far more intentional than it was.
The control did its job
Nothing went live. One downstream contract couldn't be satisfied, the work moved to blocked, and the discrepancy came back to a human. I corrected the assumption, the contract was fixed, and the work continued.
But notice: the implementation could have been perfectly correct against the plan. Testing tells me whether software matches the contract. It can't tell me whether the contract matches what I meant.
Execution controls protect system state. They don't automatically protect intent.
What I'm taking from it
- A missing ADR is a missing import. If a decision matters to the work, it has to be in the agent's context. Otherwise the model fills the gap with whatever it has.
- Separate deciding from executing. Agents can help explore a decision. An unresolved decision shouldn't silently become an implementation assumption in the same pass.
- Make the plan restate its constraints. Before execution, the plan should say which decisions it believes are in force. If it says SOPS when I meant Sealed Secrets, I want to see that before anyone writes code.
- Keep the human gate where intent matters. Not to approve every generated line, but because some decisions are organizational, architectural, or product calls that technical evidence alone can't settle.
Two different trust problems
The first failure taught me not to trust an agent's description of what it did without evidence.
This one taught me that even when execution is real, the plan coherent, and the technology valid, I still need to know which assumptions the agent resolved along the way.
One trust boundary asks: Did you actually do what you said you did?
The other asks: Did you understand what I meant?
The first can be answered with system state. The second requires explicit context.
That's what ADRs are becoming in an agentic delivery system. Not documentation the agent might read if it happens to find it.
Imports.
From the Lonely Gnome workshop
I’m Nick, building Tinkerworks. I share the engineering decisions and lessons from operating a governed agentic delivery system.
Explore the system