Who Is Accountable When the AI Agent Acts?

The agent was approved for the incident process in March. In June it changed a customer-facing configuration at two in the morning.

Nobody had said yes to that. Nobody had said no either, because nobody had been asked about that decision. They were asked about the agent.

Ask most programmes how autonomous their agents are and the answer comes back per system or per agent. Both are the wrong unit, and the wrongness is not cosmetic.

Fig 1. Enriching an event is not opening a case record, and neither is anything like issuing a determination the member can appeal.

• Per system grants the same freedom to every decision touching the CRM: dozens of decisions with nothing in common.
• Per agent forces one tier onto everything it does: set it at the riskiest task and the agent is useless, set it at the safest and it is dangerous.
  
  
• Per decision is the only unit that matches how the risk varies, and the only one a process owner can sign.

Autonomy is graded per decision, and signed off by the process owner rather than the architect.

T4 and T3 carry the lighter signature chain because the grading test has already capped them at no or low blast radius, so the risk in those rows is bounded by construction. The weight sits instead on the class boundary in code, which is what stops a T4 grant quietly widening. Where a T4 grant is proposed for anything a customer or a member can see, the cap was wrong and risk signs it too.

The most useful artefact in multi-process design, and the most often skipped, because it needs people who do not attend architecture reviews.

Fig 2. Three questions, applied to one decision. Reversibility and regulation are caps, not scores.

Two of the values in the matrix below are not binary, so the mapping has to be written down rather than assumed. Partly reversible is treated as not reversible and caps at T1. Somewhat regulated means the decision sits under a regime but produces nothing the affected party may appeal, so it caps at T1 rather than T0; the T0 cap is reserved for a determination that is itself appealable. Without that mapping the tier on a row is defensible and not derivable, which is the one thing a decision test cannot be.

Fig 3. Seven decisions run through those tests. The bottom row is a topology constraint, not a preference.

ON THE SCALE: tiers run T0, never autonomous, to T4, full autonomy, so a higher number is always more permissive.

• This is deliberately not the TM Forum autonomous networks scale, which grades the maturity of a whole network domain rather than a grant attached to one decision.
• Both ascend with autonomy, but they are not interchangeable, and a claim on one does not translate to the other.
• It is also not the write-safety rung ladder, which grades a write mechanism and the authority behind it. Rung 4 there, the first rung that commits without per-case approval, is the mechanism-side counterpart of T2 and above here. The two scales share their digits and not their meaning, so never carry a number from one into a review governed by the other.
• T2 is the tier that has to name its conditions. Conditional autonomy means the act is pre-authorised only inside a stated envelope: a named class, a change window, a value or volume ceiling, and a gate state. Write the envelope down, or T2 becomes T4 by drift.

Before the matrix is useful, the process has to have a shape.

Fig 4. The shape comes from the process, never from the framework.

The last row is the shape Fig 4 leaves out, and the payer example below uses it for claims integrity. It is absent from the six because it is a shape inside a stage rather than a shape for a whole process: a supervisor sits under one owner and one clock, and the coordination question above it is still one of the six.

The gate belongs between the agent that recommends and the executor that acts, the same boundary the control plane holds. Two structural points are easy to skip and expensive to retrofit.

Fig 5. The return path matters as much as the forward one, or the system cannot improve on what it gets wrong.

• The verification agent is not optional. A closed loop without verification is an open loop wearing a badge. Verification that reads back through the same connector the executor wrote through inherits the failure it exists to catch, so read through an independent path wherever one exists.
  
  
• Rejections are training data. An override that never lands in an evaluation set is a system that cannot improve.
  
  
• Labels only from the refusal path are a biased sample. Gated and overridden decisions are systematically the hard ones, and auto-approved decisions generate no labels at all. Route a sampled holdout of auto-approved decisions to human review purely for labelling, or the recommender is tuned on a slice of the population and drifts silently on the rest.

Cross a process boundary and the receiver needs a defined payload, or accountability evaporates at the seam.

Fig 6. Four failure modes, four contract elements. Each is discovered in production if it is not designed in.

A payer needs faster prior authorisation and better claims integrity without adding headcount, under CMS-0057-F. The operational half of that rule has been in force since 1 January 2026: 72 hours for expedited decisions, 7 calendar days for standard ones, and a specific reason on every denial. The four FHIR APIs, the Prior Authorization API among them, are due on 1 January 2027. That is why the intake diagram below still carries fax alongside a FHIR endpoint.

• Prior authorisation is a stable staged sequence, so it is a pipeline of sequential specialists passing typed payloads.
  
  
• Claims integrity is exploratory and parallelisable, so it gets a supervisor with independent threads underneath.
  
  
• Both sit inside a hard boundary: de-identified analytics and identified clinical review cannot be crossed by shared context.

None of that is the interesting decision. The constraint shaping the whole design is asymmetric autonomy: an agent may recommend approval, but may never issue an adverse determination. That is a grading, not a topology choice, and it forces the approve and deny paths into structurally different components.

WHERE THAT ROW COMES FROM: the denial path is not a preference the architect chose.

• A wave of state law enacted through 2026 says it directly. Washington SB 5395 provides that only a licensed physician or health professional may deny on medical necessity grounds, and that AI may not be the sole means used to deny, delay or modify care. Georgia SB 444 permits AI in the process but bars it from issuing an adverse determination until a qualified human reviewer conducts a review in which a clinical peer participates. Utah SB 319 and Iowa HF 2635 take comparable positions, and California SB 1120 was the earliest of them.
• Scope matters as much as the rule. Several of these take effect on 1 January 2027, so the row is fixed on a jurisdiction-specific timetable rather than uniformly today.
• ERISA self-funded plans sit outside state insurance law. For that population the row is fixed by federal rules and plan terms instead. Which is the point rather than an exception to it: a grant is scoped, and so is the constraint on one.

Fig 7. Several paths lead to approval, exactly one to denial, through a qualified human. The asymmetry is the topology.

Three non-negotiables, in the order they get violated.

• Asymmetric autonomy. Bypass must be structurally impossible, enforced at the control plane rather than in a policy PDF.
  
  
• Minimum necessary. Tool scopes are the enforcement point.
  
  
• Explainability as output. “I found nothing” and “I could not look” are different answers and must never render identically.

Three obligations the same statutes attach, which the topology has to carry rather than the policy document.

• Disclosure. Several states require a plan to state that AI is used in utilisation review, to the regulator and publicly. That makes the determination record, not a webpage, the natural place to hold it.
  
  
• Individualisation. A decision has to rest on the member’s own clinical history rather than group data alone. That is a constraint on what the criteria evaluation agent may condition on, and it belongs in the tool grant.
  
  
• Periodic audit. Accuracy and non-discrimination checks on whatever is used in review, on a named cadence, reported to a named owner. The subgroup metric below is the evidence that audit consumes.

Utilisation management and claims integrity are not one process wearing two hats, and appeals is a third.

Fig 8. Scope is the tool grant itself. Re-identification crosses back, so it is graded like any other decision.

Graded like any other decision means it earns a row on the matrix rather than a footnote at the boundary. Run it through the three questions: attaching a name to a signal cannot be undone, and in most regimes the act is regulated. Two caps fire, so re-identification lands at T0 or T1 and never above. Leave the row off the matrix and it happens implicitly the first week somebody is in a hurry.

Fig 9. The backwards reach is the design point: appeals is where you find out the other two were wrong.

An overturned appeal is evidence that the criteria evaluation, the assembly or the review was wrong. Unless it lands in the evaluation set as labelled data, the system cannot improve where being wrong costs most.

INSTRUMENT: turnaround time distribution, never the mean. Clinician agreement rate. Overturn rate on appeal. Extraction accuracy at field level. Subgroup performance for equity, reported to a named owner.

The payer example is useful because one grading in it is fixed by law. Most organisations will not have a regulator writing that row for them.

• Pick one process with a measurable baseline and a survivable failure cost.
  
  
• Map it, marking every decision point as rule-governed or judgement-governed.
  
  
• Build the deterministic spine with a human at every judgement node.
  
  
• Insert one agent, with a human reviewing every output, and measure agreement.
  
  
• Raise autonomy only on evidence, then repeat per node.

The T0–T4 tiers here grade a decision grant. They are a different scale from the write-safety rungs in When Your AI Agent Writes to Systems You Don't Own, which grade a write mechanism — the two ascend together but their digits don't line up.

How should AI agent autonomy be graded?

Autonomy is a property of a decision, not of an agent — the same agent can be T4 for one decision and T0 for another. Each decision passes through three capped questions in order: whether it can be undone cheaply and by the organisation itself (if no, capped at T1), whether it is regulated or appealable (if yes, capped at T0), and what its blast radius is, which sets the tier directly when nothing else caps it. The final tier is the lowest cap any question produces.

Who should decide an AI agent's autonomy level?

The process owner signs the grading, not the architect. Engineering can build the autonomy matrix and the gates, but deciding how much autonomy a specific decision deserves is a business and risk call that belongs to the person accountable for the process, not the person who built the system.

Why does an autonomous AI agent need a verification step?

A closed loop without verification is an open loop wearing a badge — without an independent check on what the agent did, the system has no way to detect what it got wrong. Verification should read back through a different path than the executor wrote through, or it inherits the same failure it exists to catch.

What should a cross-process handoff contract include?

At minimum: a case identifier with lineage, a resolved entity set with confidence, an evidence bundle with source references, a recommendation with stated uncertainty, an autonomy record showing what acted under which policy version, and a contract version. Miss these and accountability evaporates at the seam between processes.

Working through this for your own architecture? Blue Data Consulting builds multi-agent enterprise platforms across telecommunications, semiconductor manufacturing and healthcare. Talk to our architecture team.