Accountability in Autonomous Systems: Who Owns the Decision?

Everyone is assigning owners for AI decisions. Almost no one can hand the owner a decision to own.

“Who is accountable when an autonomous system makes a consequential call?” has settled into the rotation of open questions, the kind that gets a panel at every conference and a paragraph in every governance framework, always in the future tense, as something we will eventually have to resolve.

Except it is being resolved constantly. Every company deploying agents answers it in writing, every week: in the org chart that gives the agent a team, in the sign-off flow that puts a human before the action, in the vendor contract that caps liability, in the policy doc that names an owner per workflow. The question is not open. Somebody’s name is already on the decision.

What is actually open is a different question, one level down: what is the named person holding? Take any of those ownership assignments and look underneath it for the object it refers to. The team owns “the agent.” The reviewer owns “the approval.” The vendor owns “the model.” Nobody owns the decision, because the decision, as an artifact someone could be handed, examine, and answer for, does not exist anywhere in the system. The ownership is real. The thing owned is not.

That inversion is the piece. The hard problem in AI accountability is not assigning owners. Organizations are extremely good at assigning owners; it is what org charts are for. The hard problem is that accountability requires an object, and these systems do not produce one.

Accountability means answerability

Strip the compliance overlay off the word, and accountability means something specific: being answerable. The owner of a decision is the person who can be called on to explain it. What was decided. How the situation was read. What else was on the table. What tipped the choice. Whether, knowing all of that, the call was sound.

Which means accountability has two components that are easy to conflate. There is the obligation to answer, and there is the capacity to answer. The obligation is organizational: you can assign it, transfer it, write it into a contract. The capacity is not organizational at all. It is a property of the evidence. No policy document can create it, because it either exists in what the system retained about the moment of choice or it does not exist at all.

Human institutions rarely have to think about the difference, because in human decision-making the two travel together. The person who made the call was in the room and can be asked. And where memory is not trusted, institutions force the capacity into artifacts: the deal memo, the medical chart, the written judicial opinion. Consequential deciders are made to leave a record of their reasoning precisely so that answerability survives the moment.

Autonomous systems split the two components cleanly, and the split runs in the worst possible direction. The obligation lands on humans, necessarily; a model cannot be answerable, and sanctioning one means nothing. The capacity is available at the moment of selection, and it evaporates when the output lands. I have spent most of this newsletter on that mechanism, so one line will do here: execution is observable, decisions are not, and what a trace retains is the path taken, not the selection of that path. The evidence that answerability needs, how the input was read, what was in contention, how close the call was, is exactly the evidence the stack does not keep.

So the accountability structures being written right now share a defect that has nothing to do with how thoughtfully they assign the obligation. They assign it against capacity that was never generated. An accountability framework built this way is a set of promises to answer questions with evidence nobody collected.

Watch the chain under load

None of this is visible while things go well, because accountability is only ever exercised in the breach. So put the chain under load.

An agent with real authority, say it prices contract renewals inside negotiated bounds, makes a call that surfaces six weeks later as a problem: renewals priced defensibly wrong, in a pattern, across a quarter. The accountability chain activates, and everyone in it does their job.

The operating team produces the trace. Steps executed in order, tools returned cleanly, no errors, nothing anomalous. The reviewing human produces the approval record: at sign-off, the numbers sat inside the negotiated bounds and looked consistent with policy. The vendor produces the evals: the model performs within expectations on the benchmark suite. The policy owner produces the policy: the written rule was correct as written.

Every one of them answered. Now look at what each answer was actually an answer to.

| Who gets asked      | What they can produce | What their evidence answers         |
|---------------------|-----------------------|-------------------------------------|
| The operating team  | The trace             | Did the steps execute correctly     |
| The reviewing human | The approval record   | Did the output look acceptable      |
| The model vendor    | Evals and benchmarks  | Does the model perform in aggregate |
| The policy owner    | The written policy    | Was the rule defensible as written  |
| No one              | Nothing               | Was this decision sound             |

The question the incident actually raised, was this decision sound and who should have caught it, was never answered by anyone, because it never attached to any evidence anyone held. It got decomposed into four adjacent questions, each answerable, none of them it.

This is the signature failure mode, and it is worth naming precisely because it does not look like failure. Nobody stonewalled. Nobody’s evidence was wrong. Accountability in these systems does not break. It disperses.

And a question that cannot attach to evidence does not stay open, because organizations abhor an open incident. It resolves to one of two defaults. Either it lands on the nearest human, usually the reviewer, whose approval is the last place a person visibly touched the outcome. Or it lands on no one: the postmortem closes with a narrower rule and a new guardrail, and the mechanism that produced the decision is left exactly as answerable as it was before, which is to say not at all. I walked that second ending in the silent-failure piece. The first ending is worse, and it is next.

The human in the loop is a co-signer

Human-in-the-loop is the load-bearing answer in nearly every accountability framework I have seen: a person approves consequential actions; therefore a person owns them. It is the mechanism that lets the ownership chart close. It is also the clearest case of obligation assigned without capacity, because you can enumerate exactly what the approver is shown.

They see the output. Maybe a diff, maybe a summary, maybe the top of the supporting context. Here is what they do not see, and cannot, because it was never captured: how the system read the situation, which alternatives were live, how close the call was, and which of the governing signals actually figured in the choice versus merely sat in the context. The primitives of the decision, in other words. The approver is being asked to underwrite a selection while being shown only its result.

There is a word for adding your name to a risk you did not originate and cannot inspect, and it is not “reviewer.” It is co-signer. A co-signature changes who pays when things go wrong. It does not change whether anyone understood the risk. A human in the loop who can see the output but not the decision is not exercising oversight. They are terminating a liability chain, and most of these loops are shaped as if that were the goal.

This is also why approval fatigue is not a discipline problem, much as it gets treated like one. When approval cannot engage the decision, it degrades into pattern-matching on outputs, and outputs are precisely where these systems look most fine. Locally coherent, every time; that is what makes the failures silent. An approver who waves through a hundred clean-looking outputs is not being lazy. They are doing the only version of the job the evidence in front of them permits.

So the loop does something subtler than fail. It moves blame downward while leaving answerability nowhere. The human absorbs the liability. The judgment, the thing accountability was supposed to reach, stays exactly where it was: inside a selection process nobody recorded.

The auditors are arriving at the same place

Regulatory frameworks are beginning to expose the same tension from the outside. The ones now coming into force for consequential automated decisions, the EU AI Act most visibly, keep circling two requirements: records sufficient to trace the system’s operation, and human oversight that is effective rather than nominal. Set aside the compliance mechanics and look at the diagnosis embedded in that language. Regulators did not need to be told that a human stapled to the end of a pipeline is not oversight. They wrote “effective” into the requirement because the nominal version is what everyone builds by default.

The catch is that the record-keeping half is being satisfied with the records that exist, and the records that exist are execution records. A system can retain every log line, replay every run, and pass a records audit in the literal sense while remaining unable to say why it chose anything. Which means the two requirements are quietly in tension: the oversight cannot become effective on the evidence the record-keeping produces. Sooner or later, an audit asks the same question the incident review asked, and it disperses the same way.

Ownership follows evidence

The question in the title does have an answer. It is just an answer to a slightly different question than the one people mean when they ask it.

Who owns the decision? Whoever can be handed it. Ownership follows evidence. It always has; that is what the deal memo and the medical chart and the written opinion were for. Institutions that needed decisions to be owned never relied on memory or good intent. They made the decision durable enough to examine, and ownership attached to the artifact. Not because writing is sacred, but because answerability needs an object, and an object is the one thing an org chart cannot conjure.

Until the decision exists as that kind of object, structured, inspectable, comparable against the alternatives that were live at the moment of choice, “who owns the AI’s decision” will keep resolving to its two degenerate answers: the nearest human, or no one. Assigning the obligation harder does not help. Obligation without capacity is not accountability. It is liability with better formatting.

I wrote a few pieces back that “the model decided” is no longer an acceptable end of a sentence. It turns out “a human approved it” is not one either, and for the same reason: both sentences stop exactly where the evidence stops. The decision has no owner yet because there is nothing yet to own. Produce the decision as evidence and the ownership question stops being philosophical. It becomes what it has always been in every other consequential domain: a records question, with a records answer.