Yesterday on LinkedIn, I wrote about JPMorgan and Schwab refusing to let outside AI agents move client money. The part that stayed with me wasn’t the refusal. It was what a bank’s records would show if they allowed it: a session authenticated as the client, and the transfer that session executed. Both would be accurate. Neither would say who chose the transfer.
That pattern is bigger than banking. A modern agent stack is full of systems that check something and say yes, and every one of those yeses is real. The trouble starts when a team treats a pile of them as an answer to a question none of them was asked.
A stack of yeses
A composite, to show the shape rather than report a case.
A company of about sixty people uses an agent to handle employee travel changes. When a flight is cancelled, the employee messages the agent and it rebooks them. The operations manager owns the workflow, set up the travel policy it runs under, and is the person who hears about it when something goes wrong.
On a Tuesday evening a consultant’s flight to Chicago is cancelled. She messages the agent: she has a client meeting at nine the next morning and needs to be there for it. The agent searches, finds a 6:10 a.m. flight that lands at 7:55 and a 9:05 flight that lands at 10:40, and books the 9:05. The earlier one was over the fare cap.
She sees the itinerary at eleven that night, books the 6:10 herself, and files an expense exception in the morning from the gate.
Now go back through what the company’s systems recorded about that booking.
The agent acted under her travel profile, through the company’s booking account, with her saved preferences. As far as the airline and the booking platform were concerned, the request came from her. Identity said yes.
Before the booking went through, the company’s policy check evaluated it. Approved carrier, economy, under the fare cap, inside the advance-booking rules. The check logged its verdict: allowed. Policy said yes.
The booking platform returned a confirmation that matched the approved itinerary exactly: same flight, same fare, same traveler. Nothing was changed between approval and purchase. Execution said yes.
On Friday the operations manager did their weekly spot check of a sample of agent bookings, and this one was in the sample. Right carrier, right fare class, right date, under the cap. They marked it fine. Review said yes.
Four systems, four yeses, all of them accurate. The consultant still landed an hour and forty minutes after her meeting started, and she only made the meeting because she overrode the agent at eleven at night.
Every yes records someone else’s decision
The tempting reading is that one of those layers failed. I don’t think any of them did. Each answered its own question correctly. The problem is whose decision each one recorded.
| The yes | Whose decision it records | The question it answers |
|---------------|-----------------------------|----------------------------------------------------|
| **Identity** | the authentication system's | Was this session allowed to act for her? |
| **Policy** | the policy check's | Was this action permitted under the rules? |
| **Execution** | the booking platform's | Did what ran match what was approved? |
| **Review** | the reviewer's | Did anything in the record look wrong to a person? |Read down the middle column. Every row is a real decision, made by a real system or person, and every one of them is logged. The one decision with no row is the agent’s: given what she asked for and what was available, it chose the 9:05.
That’s the decision the consultant would ask about, and so would the operations manager the moment the expense exception lands. Why did it do that? None of the four records can answer, because none of them was written about that choice. The reviewer is the instructive case. The operations manager looked at the booking and saw nothing wrong, because what they were shown was the booking. The message that mattered, “client meeting at nine,” wasn’t on the page they reviewed.
## The policy verdict is the strongest yes
Of the four, the policy layer deserves the most serious treatment, because it’s where a lot of current work on agent safety is going, and the work is good.
A runtime verdict is evaluated at the moment of action, against a written rule, by something other than the agent. It’s logged. It can carry a confidence value an application can act on. In the better versions, when a reviewer corrects a verdict, the correction is kept and informs the next one. For a regulated team the question it answers, was this permitted?, is often the first question an auditor asks, and before these products existed many teams couldn’t answer it at all.
A verdict is one of the best pieces of evidence a team can have about an agent’s action. It was captured at the time, by a system the agent doesn’t control, about a rule a person wrote. That is close to the gold standard for where evidence comes from.
It’s still a verdict about the action measured against the policy. The rebooking was permitted, and it was the wrong rebooking. Both statements are true at once, and only one of them is in the log. A policy engine could reasonably grow toward the second question over time, by attaching what the user asked for and what happened afterward. My point is about what the verdict, as a record, was built to hold. Today it holds whether the action cleared the rules.
Which means the useful way to see a verdict is as a source: one high-quality input to the account of a decision, and the strongest input there is for one specific question.
What would answer the question the yeses can’t
On this channel the real question is about objects: what would have to exist for “why did the agent book the 9:05?” to have an answer the following Friday?
Something has to be about the agent’s decision specifically, rather than about a session, a rule check, a purchase or a review. It has to hold what she asked for, in her words, because “need to be there for it” is the constraint every other layer missed. It has to hold what the agent had to choose from, because “why the 9:05?” really means “why the 9:05 and not the 6:10?” Then it has to hold the yeses themselves, the verdict included, as evidence, each one labelled with what it actually establishes. And it has to leave room for what happened afterward.
Put those together and the separate questions become visible as separate questions:
- Was it allowed? Yes. The policy verdict establishes that.
- Was it executed as approved? Yes. The booking confirmation establishes that.
- Did it serve what she meant? Probably not, but that’s an inference from her message and the arrival time, and it should be filed as one.
- What happened afterward? She rebooked herself and filed an exception.
Each of those can come back unknown, and that’s legitimate. In plenty of real cases nobody will know whether the agent served the request, because the request was ambiguous or the outcome never got reported. A record that says “unknown” there is more useful than one that says nothing, because it tells the next person exactly which question is still open.
The temptation, once you have four answers, is to compress them into a score. Allowed, executed, probably-not-served, reopened: call it a 6 out of 10. The score would look reasonable on a dashboard, and it would hide the one thing worth knowing, which is that the strong answers and the failure were about different questions. A high “allowed” doesn’t offset a low “served.” They don’t add up.
The reopened case doesn’t settle it either
The outcome is evidence, and it’s easy to over-read.
She rebooked and filed an exception. That looks like proof the agent was wrong. It might not be. The 6:10 was over the cap, and the company set that cap for a reason. Maybe the right behavior was to stay in policy. Maybe it was to ask her: “The flight that gets you there by nine is over the cap. Do you want me to request an exception?” Maybe the meeting moved and she’d have rebooked anyway. The outcome tells you the agent’s choice didn’t hold. It doesn’t tell you which of those was the better call.
What it does is narrow the investigation. Combined with her message and the two flights, the outcome points at a specific question the team now has to answer about its own design: when the rules and the request conflict, is the agent supposed to pick the rules silently, or say so? That’s a decision about the workflow, and it can only be made by people looking at the agent’s decision with the yeses and the outcome next to it.
Where this leaves a small team
Most small teams running agents don’t have a runtime policy layer. They have an API key, which is the identity yes, and execution logs, which are the execution yes, and someone who looks at a sample when they have time.
The tempting conclusion is that the fix is to add the missing layers. Adding a policy check would give that team a real, useful fourth yes. It wouldn’t have caught the 9:05, because the 9:05 was inside policy. The gap isn’t a missing layer of checks. It’s the missing account of the decision the checks were all checking around.
That changes what the first investment looks like. Before another layer that says yes, a team needs somewhere for the agent’s actual choices to land, with what was asked, what was available, which yeses it passed, and what happened afterward. The yeses then become what they’re good at being: evidence for specific questions, from sources the agent doesn’t control.
The research conclusion is narrower than “checks don’t work.” Every layer in an agent stack records a decision. Almost none of them records the agent’s. A stack where every check passed has shown that the action was authorized, permitted, executed and reviewed, and it still hasn’t shown whether the agent served the request it was given. That question needs its own record, and the yeses belong inside it as evidence.
If you’ve had an agent pass every check you had and still get it wrong, I’d like to hear the shape of it. Reply here; I’ll keep anything you send unidentifiable.
That record, the place where the yeses meet what actually happened, is what I’m building Nalyqor to keep.