Every reliability tool has a first screen, and the first screen is an argument. It tells you what the people who built it think reliability is.
A service dashboard argues that reliability is aggregate health: error rate, latency, saturation, a line that should stay flat. An eval suite argues that it’s a pass rate you establish before you ship. A governance policy has a first screen of its own, a list of controls, and it argues that reliability is something you declare and then audit.
Each is a good argument for the system it was designed around. For production AI I think they share a problem. They put a summary in front of a person who usually arrives holding one specific case.
When an AI system first becomes unreliable in a way that matters, it often shows up not as a number moving but as a question about one decision, asked by someone who may not work on the system at all: why did it do that?
The last two pieces here took that question for granted. The Operator Gap was about who goes missing when a workflow outgrows its team, and last week’s piece was about why the operator’s investigation is where governance evidence starts. This one looks at the question itself, as an interface: the thing a person brings to a system, and the thing the system has to be able to answer. The part that belongs on this channel is what answering it takes, because a question turns out to be a much stricter interface than a dashboard.
How it actually arrives
Here is a composite, built to show the shape rather than report a case.
A software company of about forty people uses an agent to route inbound leads. It reads the demo-request form, enriches the company, and sends each lead one of three ways: to an account executive, to a self-serve trial, or to a nurture sequence. The routing guidance says that larger companies showing a buying signal go to sales.
Three weeks after a director of operations at a 600-person logistics company filled in that form, one of the account executives sees the company announce a contract with a competitor. They search the CRM and find the lead. It went to nurture and received two automated emails.
Nothing on any dashboard was off. The agent ran on every lead that week, latency was normal, no tool call failed, and the month’s conversion rate sat inside its usual range. One lost deal is noise in a monthly conversion rate and a real problem in one account executive’s quarter.
The account executive doesn’t want the conversion rate. They want to know why that lead went where it went. So does the head of sales, who now has to decide whether to keep trusting the routing, and so does the founder, who approved it.
Look at who is asking and how. The first reliability event for this system wasn’t an alert. It was a person outside engineering, holding one record, asking a causal question in plain language. For agent failures that complete successfully, that’s a common shape, and it’s the case reliability tooling for AI has to meet first.
A question needs an object the dashboard never did
The dashboard needed events to exist: runs, timings, token counts, tool calls, errors. They’re cheap to emit and easy to aggregate, and none of them has to know what a decision is.
“Why did it go to nurture?” asks for a good deal more.
It presupposes that the routing decision for this one lead can be found as a thing in its own right, rather than dug out of a run that also did enrichment, deduplication and a few other steps. Then you need what the agent had in front of it at that moment: what the form said, what enrichment returned that morning, and which version of the routing guidance was in effect. (The guidance had been edited that month, so which version applied is the first thing anyone will ask.) You need what the agent could have done instead, because “why nurture?” really means “why nurture and not sales?” And you need some way to tell an answer from a guess.
None of that is exotic. Nearly all of it was available at the moment the agent chose. Very little of it is still around three weeks later.
This is why the choice of first interface matters more than it looks. Interfaces shape the data behind them. A team that starts with a dashboard builds the event pipeline a dashboard needs, and it can run that way for a long time, because an aggregate never complains about what it can’t see. A team that starts from the question finds out on the first afternoon that it can’t answer it, and what’s missing is the decision itself as a first-class record. The question forces the object into existence. The dashboard can get along without it indefinitely.
The metric comes after the question
The obvious objection is that questions are anecdotes. Reliability engineering exists to get past anecdotes to the rate, the distribution and the trend, so why start with the anecdote?
That objection is right about where reliability ends up. It skips a step about where it starts, and the step is specific to systems that exercise judgment.
An aggregate can count a failure mode once something has defined or instrumented it as worth counting. An error-rate chart works because much of the failure vocabulary for a conventional service was settled before anyone drew it: an exception, a timeout, a 500. For a routing agent, what’s the equivalent? “Misrouted lead” isn’t a field anywhere. Before the account executive’s afternoon, nobody on that team had described “large company, personal email address, no enrichment match” as a case worth watching. Anomaly detection helps when a distribution moves, but one lead sent the wrong way doesn’t move any distribution, and even a flagged shift still needs someone to say what it means. You can’t put a line on a chart for a failure mode nobody has described yet.
A dashboard counts the failures you’ve already named. A question is how you find the ones you haven’t.
That’s the case for “first,” as distinct from “only.” Answered well, a question leaves behind a described failure: this kind of input, read this way, producing this outcome. Described failures are what categories are made of, and categories are what eventually become worth counting. In From Observability to Behavioral Intelligence I wrote about what a population of decision records makes possible. This sits upstream of that. The population’s vocabulary, meaning what to group by and what to watch, comes out of individual questions answered one at a time. Start from the aggregate and you’re guessing that vocabulary in advance.
The easiest answer is the wrong one
If the question is the right interface, the cheapest way to build it is obvious. The agent is a language model. Ask it.
It will answer, and the answer will read well. Something like: ”The lead was routed to nurture because the contact used a personal email address and described themselves as ‘just exploring,’ which suggested an early-stage inquiry.” It’s specific, plausible and in plain English, which is exactly what the head of sales asked for.
It’s also a new generation, produced three weeks later, possibly by a different model version than the one that made the call, reasoning about whatever it’s being shown now. An agent’s account of its own behavior is one more output. It can be right. It can’t serve as its own evidence.
Here is the same question answered from a record instead:
Observed: the lead was routed to nurture on the 3rd at 10:42. The form listed a 600-person company and a personal email address. Enrichment that morning returned no company match.
Reported: the application logged which version of the routing guidance was in effect. It had been edited two days earlier to add a line about leads with personal email addresses.
Inferred: that new line, applied to a lead with no enrichment match, is the likelier reason it fell short of sales. The “just exploring” wording looks secondary. Not yet checked: how many other leads with personal addresses went the same way that week.
It’s less fluent. It’s also a different kind of thing. It keeps what was seen apart from what the application said about itself and from what someone concluded. It points at a specific change. It says what it doesn’t know. And in this composite it disagrees with the agent’s own account on the point that matters, since the agent’s story was about wording and never mentioned the guidance edit at all.
So the interface claim needs a condition attached. “Why did it do that?” is the right first question, and whether a tool answers it from a record or by asking the agent decides if it’s a reliability interface or a chat window.
What “first” leaves unsolved
Starting from the question has a selection problem, and it’s the one The Operator Gap was about: you only ask about decisions that surfaced. This lead got asked about because a competitor announced a contract. Whatever other leads went the same way that week did not, and a question-first interface on its own will never find them.
The way out runs through the first answer rather than around it. Once the no-match case is described, the team knows what to look for among decisions nobody asked about. That only works if the answer was kept as a record, in a form the next decision can be compared against, which is the argument last week’s piece made from the governance side.
The question doesn’t replace evals or monitoring either. Evals still tell you whether a change is safe to ship. Monitoring still tells you the system ran. Neither was built to answer a causal question about one production decision, and neither is wrong for that.
And a single answer is an estimate. The inferred line above could be wrong. What makes it useful is that its uncertainty is attached to one specific claim somebody can go and check.
The conclusion I’d draw
The research conclusion is about order. Reliability tooling for conventional software could start from the aggregate because so much of the failure vocabulary existed before the dashboard did. For systems that exercise judgment, that vocabulary mostly doesn’t exist until someone asks about a specific decision and gets a real answer back. That makes the question the first interface. It also makes the decision record the first object, because the question can’t be answered honestly without one.
Dashboards, evals and governance still come, and they come better, because by then they’re counting failures somebody actually described.
“Why did it do that?” is the question I’m building Nalyqor to answer first, from a record the agent didn’t write.