A Human in the Loop Is Not a Control

Many control diagrams end with a reassuring box: human review. The label suggests that judgment has returned to the process just before consequence. Yet the diagram rarely says what the reviewer can see, how much time they have, which failures they are qualified to recognize, or whether they can actually prevent the next action.

A person asked to approve a fluent recommendation every few minutes may occupy the loop without controlling it. If the system has already assembled the evidence, framed the alternatives, and presented one answer as normal, disagreement carries both cognitive and organizational cost. The easiest action is to accept.

For a board, the important question is not simply whether a human appears in the workflow. It is what must be true for that human to interrupt the workflow when the machine is confident and wrong.

The International AI Safety Report 2026 notes that expert human oversight can mitigate some reliability risks. It also describes automation bias: people may trust automated output more than warranted, overlook contradictory information, or act on incorrect advice. The strength and persistence of that bias vary with the task, interface, and accountability.

This is why “keep a human in the loop” is too broad to be a safety conclusion. A skilled reviewer with good evidence and time may catch an error. An overloaded reviewer facing hundreds of apparently routine recommendations may add delay while contributing little discrimination. Humans are not automatically safer than AI, and AI is not uniformly less capable than its reviewers.

Oversight design must therefore examine the human-machine system rather than assign virtue to one side of it. What matters is whether the combined arrangement can detect, contain, and recover from the failures that matter in that context.

Article 14 of the EU AI Act applies to high-risk AI systems within that regulation. It requires systems to be designed so they can be effectively overseen by natural persons, with measures commensurate to risk, autonomy, and context. The legal scope is specific; it should not be paraphrased as a universal rule for every AI deployment.

Still, its list is instructive. As appropriate and proportionate, overseers must be enabled to understand relevant capabilities and limitations, monitor for anomalies and unexpected performance, remain aware of automation bias, interpret outputs, decide not to use or to override them, and intervene or stop the system safely.

Those are operational verbs. They move oversight beyond a named person and toward an actual capability. They also reveal a dependency boards should ask about: who possesses enough expertise to perform those verbs, and how will the organization know that capability is still present six months after automation becomes routine?

A reviewer does not need to outperform the system on every routine case. They do need enough domain competence to recognize when the output has crossed a consequential boundary. That may require understanding weak signals, common failure patterns, the provenance of key inputs, and the conditions under which an apparently reasonable answer should be distrusted.

Minimum viable expertise is not a credential alone. It is capability relative to the decision. A licensed professional may lack local context; a long-tenured operator may know the context but lack authority; a risk officer may understand consequence but be unable to inspect the evidence that produced the recommendation.

The threshold should rise with consequence, irreversibility, poor observability, and limited recourse. It should be defined before deployment, then tested through cases in which the system is deliberately wrong or the evidence is deliberately incomplete.

Even a capable reviewer cannot judge what the interface conceals. Oversight needs the right evidence at the moment of review: source material, uncertainty, material assumptions, prior actions, and the reason the case was escalated. A single confidence score can narrow attention without explaining where the system may be brittle.

Workload matters just as much. If review volume makes careful attention impossible, the formal control may decay into sampling, habit, or approval fatigue. The organization should know the expected queue, the time available per consequential case, and what happens when demand exceeds capacity.

Automation can also reduce the reviewer’s exposure to the very cases that once maintained expertise. The evidence on long-term deskilling across enterprise settings remains incomplete. Still, the possibility is sufficient to justify periodic unaided practice, exposure to edge cases, and tests that ask whether the human can still recognize failure without the system framing it first.

An override button is only as real as the organization behind it. Reviewers need explicit decision rights, protection from retaliation for appropriate intervention, and an escalation path when commercial pressure favors speed. They also need a safe system state: stopping an automated process should not create a second unmanaged risk.

The economics create a genuine tension. Extensive expert review can erase the time and cost advantage that justified automation. The answer is not universal full review. It is risk-based design: constrain permissions, automate reversible low-consequence work more freely, and concentrate strong human control where failures are hard to observe, hard to reverse, or costly to remedy.

When the organization repeatedly overrides the same class of case, that pattern should feed back into evaluation and system design. When nobody overrides anything, leaders should resist congratulating themselves. Perfect system performance and ceremonial oversight can look identical in an approval log.

A credible control has test results, not only a workflow chart. Give reviewers representative failures, subtle anomalies, and cases in which the correct action is to seek more information. Measure detection, escalation, response time, and the quality of the intervention. Include situations where accepting the AI is correct so the exercise does not train reflexive rejection.

Governance should also name who maintains the evaluation set, who refreshes it as systems and conditions change, and who owns residual risk. The reviewer’s competence, workload, and authority are operating assumptions. They should be revisited with the same seriousness as model performance and access controls.

This article is authority-building research, not an assertion that Skagway provides broad AI-governance consulting. The connection to Skagway’s current work is narrower: when a critical expert is leaving, an organization may need to preserve enough judgment behind a genuine escalation path. Legal interpretation, regulatory compliance, AI assurance, cybersecurity, and system validation belong with qualified specialists in those fields.