Observability earned its place. That is worth saying plainly, because the argument I want to make gets misread as a complaint about it, and it is not.
Before observability, a production system’s internals were genuinely dark. You had metrics, which told you the aggregate was healthy, and you had logs, which told you what one thing printed. Between them sat an enormous gap, and the way you crossed it was to guess. What changed was not volume. Teams already had plenty of data. What changed was that executions became high-cardinality objects you could interrogate after the fact, which meant you could ask a question nobody had thought to instrument for in advance. That is the whole trick, and it was a real one. The industry moved from “is the system healthy” to “what exactly happened on this request,” and it moved there in about five years.
That capability transferred directly to AI systems, and it works. I can trace an agent run end to end, see every tool call, replay the sequence, and decompose a failure into steps. The tooling is good. The engineers who built it were solving a real problem and they solved it.
Its primary unit of investigation is an execution: the request, the span, the trace, the event.
That is not a criticism, it is a design fact, and for the systems observability was built for it was the correct choice. In conventional deterministic application logic, the same state and inputs are expected to produce the same result. The execution path therefore tells you a great deal about the behavior you should expect to see again.
Probabilistic systems break that equivalence, and they break it at the root.
The execution is no longer the behavior
Run the same agent against the same input a hundred times and you can get a hundred slightly different runs. Most of them fine. Some of them different in ways that matter. No single one of them is the system’s behavior. The behavior is the shape of the whole hundred: what it usually reaches for, how tightly it clusters, where the tail sits, and how all of that shifts when the context changes underneath it.
This is why the most common instinct is the wrong one. When a team cannot explain what their agent did, the reflex is more instrumentation: capture more of each run, log the intermediate reasoning, retain more context. That reflex is correct in a deterministic system, where the missing explanation really is hiding in some field nobody captured. Here it produces a richer description of one sample from a distribution you still have not characterized.
You can see the mismatch in what aggregation currently gives you. Error rate, p95 latency, token spend, tool-failure counts: those are real aggregates over many runs, and every one of them aggregates execution. They describe the machinery. There is no standard production primitive for judgment equivalent to error rate or latency, and the reason is structural rather than a gap in anyone’s roadmap. Aggregating something operationally requires it to be an object first, and judgment has not become one. I spent a whole piece on what it would take to make it one, so one line will do here: a trace reliably retains the path taken, and by itself it does not turn the selection of that path into a comparable decision object.
So the stack has an aggregate view of execution and an instance view of nearly everything else. Both halves are load-bearing, and neither is the thing production teams keep reaching for.
The questions being asked are comparative
Watch what someone actually asks when an AI system does something strange in production. Almost none of it is answerable about a single run.
Is this normal for this agent? Has it done this before? Did something change, or has it always been like this and we only just noticed? Is this one workflow or is it everywhere? Is it getting worse?
Every one of those is a comparison against a population. Not one of them can be answered by looking harder at the run in front of you, no matter how much of that run you captured.
| Question | Where it can be answered |
| ----------------------------------------------- | ------------------------- |
| Did the request succeed | Execution |
| Which steps ran, in what order | Execution |
| How was this input read, and what else was live | The decision, as a record |
| Was this particular choice sound | The decision, as a record |
| Is this normal for this system | A population of decisions |
| Has this failure shape occurred before | A population of decisions |
| Is judgment degrading, and since when | A population of decisions |The bottom three rows are where production teams live, and they are the rows with nothing underneath them today.
What a population gives you that a record cannot
Consider an agent that triages inbound requests and decides which ones escalate to a human. Pick any single escalation it declined and inspect it properly, with the decision reconstructed rather than inferred: how the request was read, which alternatives were live, how close the call was, which governing signals actually figured in it. The call may look perfectly defensible on its own.
Now hold that same decision point still and look at every instance of it over the past two months. A different object appears. The margin between escalate and resolve has been narrowing steadily. A category of request that used to sit clearly on the escalate side now lands just barely on the other. Nothing broke. No error fired. Every individual decision in that drift was defensible on its own terms, which is precisely why nobody caught it, and why the first visible symptom arrives as a customer complaint rather than an alert.
Three things become available across a population that are unavailable in any single decision, no matter how well captured.
A baseline. You know what normal latency looks like for your service. You almost certainly do not know what normal judgment looks like for your agent: how decisive it usually is at a given decision point, which signals usually drive it, how wide its considered set typically runs. Without that, “unusual” is not a claim anyone can make, and every anomaly assessment is somebody’s intuition.
Drift as a first-class signal. Slow failures are often hard to see at the individual-output level because no single output looks obviously wrong. They are visible as distribution shift: margins tightening, a decisive signal quietly dropping out of contention, a stale assumption recurring across runs that should have decayed. You see judgment degrade before answers do.
Recurrence. This is the one teams underrate most. Right now, when an agent fails in a way that requires real investigation, that investigation starts from zero. The engineer reconstructs what happened, reaches a conclusion, fixes something, and the reconstruction evaporates. Six weeks later the same failure shape appears in a different workflow and the next engineer starts from zero again. The team is not learning how the system behaves. It is repeatedly rediscovering it, at senior-engineer rates, and paying the full cost every time.
That last one moves the problem out of debugging and into organizational memory, which is a different category of thing entirely.
A different object, not a better dashboard
It would be easy to hear this as an argument for adding behavioral charts to an observability product, and the distinction is worth drawing carefully, because it is not a claim that anyone is technically incapable of building it.
Observability platforms were built around an engineer who reads traces. Eval frameworks were built around someone who writes them. Both assumptions are reasonable and both describe people who build AI systems. The questions in the bottom three rows of that table are asked by a different person: whoever is responsible for what the system actually does once it is running, which is frequently not the same person and frequently has no ML engineer to hand the question to.
The deeper difference is the object of intelligence. Observability’s object is execution, and it is genuinely good at it. The object here is judgment: not whether the machinery ran, but whether the system reasoned, chose, retrieved, and adapted in a way that made sense, and whether that has been holding steady or quietly moving.
Any of this could live inside the same product someday, and an existing platform could extend its data model to reach it. What it cannot be is another chart drawn over the same primitive. You have to make the decision a first-class object before a population of decisions exists to analyze, and that is a change to the object model rather than an addition to the interface. Every trace you already collect stays exactly as useful as it was, which is the part I want to be unambiguous about. Nothing here argues for ripping anything out.
Behavioral intelligence is the term I will use for the capability that appears once decisions accumulate: the comparative, accumulated view of how a system decides, built from decision records evaluated as a population over time. I hold the label loosely, as I have from the start, because the market picks the word that names the problem people feel and nobody assigns that from the outside. The capability is the durable part. What it has to deliver is a baseline for judgment, drift you can see before outputs go bad, and memory that survives the engineer who did the last investigation.
It is also worth being clear about the limits. This does not make decisions directly observable, because nothing does. It does not tell you whether a business outcome was correct; that judgment sits above this work and stays with a human. It will not give you certainty about any single decision, since a reconstructed decision is an estimate with a derivation, not a readout. What changes across a population is that estimates become comparable to each other. One reconstruction is an estimate. Hundreds of comparable estimates moving together over time become evidence of a pattern.
When decisions become a population
There is a pattern in how this stack has grown, and it is not that each stage collects more data than the one before it. Metrics existed long before observability, and teams drowning in logs did not lack volume. What changed each time was the unit: the metric gave us the aggregate, the trace gave us the request, and each time the unit changed, a class of question became askable that had previously required a guess.
The decision is the next unit. Behavioral intelligence is not a further unit stacked on top of it, and I want to be careful not to imply otherwise. It is what a population of that unit makes possible, in the same way a single trace is interesting and a corpus of traces is operational. The decision record is the atom. The behavior is what the atoms show you when there are enough of them to compare.
Probabilistic systems are going to keep being probabilistic. That is not a defect to be engineered away, it is the property that makes them useful, and the teams shipping them are not asking for determinism. They are asking for dependable outcomes from a system that will never be deterministic, which is a different and much more tractable request. Dependability under uncertainty has never come from eliminating variance. It comes from characterizing it: knowing the distribution, knowing where the edges are, and noticing early when it moves.
That is the capability the AI stack does not have yet, and the reason it does not have it is that the thing it would need to characterize has never been treated as an object.
Execution tells us what happened. A decision record tells us why this happened. A population of those decisions tells us what normal looks like.
I am building Nalyqor across those last two: reconstructing decisions as records, then treating them as a population that accumulates. The bet is that this is where the stack goes next, and that the teams who need it soonest are the ones running consequential AI without anyone whose job is watching how it decides. That part is a thesis, not a finished argument. The diagnosis I am confident about: the questions production teams keep asking are comparative, and comparative questions need a population nobody is currently keeping.