What Should Never Be Delegated to an AI Agent?

Leaders often ask for a clean boundary: which decisions must remain human? A list promises moral clarity and an implementable policy. It also ages badly. The same task can be advisory in one organization, transactional in another, and physically consequential in a third. Model capability, system permissions, law, and available recourse all change the risk.

The enduring question is what the organization cannot yet delegate safely under its actual conditions. That wording leaves room for capability to improve while keeping attention on consequence and governance. It also prevents a technically impressive demonstration from becoming the only evidence used to expand autonomy.

A board does not need to predict which parts of judgment will remain uniquely human forever. It needs a defensible present boundary, evidence for that boundary, and a process for changing it without outrunning the organization’s ability to detect failure.

Two systems may perform the same classification while carrying radically different authority. One drafts a recommendation that a person can revise. Another initiates a payment, changes a customer’s access, or directs equipment. The output may be identical; the consequence path is not.

Ask what a wrong decision can harm: health, safety, rights, capital, operations, reputation, or a person’s ability to obtain remedy. Consider both the single event and cumulative scale. A low-value action repeated thousands of times can create a material exposure that no individual decision reveals.

Consequence does not automatically prohibit delegation. It determines the strength of evidence, constraints, monitoring, and human authority required before delegation becomes reasonable.

A reversible action gives the organization time to learn. A draft can be discarded. A recommendation can be challenged. A temporary allocation may be corrected before anyone materially relies on it. Irreversible or path-dependent actions deserve a different posture because later correction cannot restore the original position.

Recourse asks whether the affected party can see, contest, and correct the decision. Internal operators may have a direct escalation route; customers, employees, or citizens may not. A formally reversible decision can be functionally permanent when the person harmed does not know it occurred or cannot reach anyone with authority.

The delegation boundary should therefore account for the full path from action to remedy. “A human can reverse it” is weak assurance if detection arrives months later or reversal requires exceptional effort from the person affected.

Some failures announce themselves quickly. A transaction rejects, a test fails, or a customer reports the problem. Other errors look like success: a missed risk, a subtly unfair ranking, a plausible diagnosis that suppresses further inquiry, or a relationship decision whose cost appears only later.

Low observability makes performance statistics easier to misread. The organization sees completed tasks and few reported incidents, then interprets silence as reliability. Representative evaluation, delayed-outcome review, and channels for external challenge become more important as feedback weakens.

The International AI Safety Report 2026 describes an evaluation gap between controlled tests and behavior in dynamic real-world settings. The implication is not that deployment can never proceed. It is that pre-deployment performance should not be treated as permanent permission.

A system cannot be said to escalate well unless someone can define the conditions that warrant escalation and judge the case after it arrives. The organization needs an independent benchmark: observable outcomes, authoritative rules, qualified human judgment, or another validated source that does not merely echo the same model output.

Article 14 of the EU AI Act offers a concrete reference for high-risk systems within its legal scope. It calls for overseers who can understand relevant limitations, recognize anomalies and over-reliance, interpret output, disregard or override it, and stop the system. NIST’s voluntary AI Risk Management Framework similarly treats governance, context mapping, measurement, and management as continuing lifecycle work and asks organizations to define human roles and oversight processes.

These sources do not create one boundary for every use. They make a common weakness visible: naming a human owner is insufficient when the person lacks information, competence, time, or decision rights.

Start with constrained permissions and cases whose failure is observable and recoverable. Test representative situations, altered cases, adversarial inputs, and escalation behavior. Increase authority only after the full system—including tools, data, interfaces, and people—has demonstrated performance within the intended context.

The evidence threshold should rise as consequences become harder to reverse and failures harder to see. Expansion should be conditional: a change in model, data source, workflow, or environment can move the system outside the evidence that supported the prior boundary.

The opposite movement matters too. A governance design needs a way to reduce autonomy after incidents, drift, weak oversight results, or loss of expert benchmark capability. Delegation is an operating state, not a one-way maturity ladder.

A vendor can describe capability and supply evaluations. It cannot decide the organization’s tolerance for harm, the legitimacy of a decision right, or the remedy owed to people affected. Those choices belong to governing and operating authorities with appropriate legal, risk, technical, and domain advice.

Skagway does not currently provide broad AI-governance or AI-agent validation services. A connection to current succession work exists only when a named critical leader or expert owns the decision boundaries and escalation judgment the institution is at risk of losing. In that case, preserving those boundaries may support continuity, but it does not replace technical assurance or compliance work.

The board’s durable question is therefore less dramatic than “what must always remain human?” It is more demanding: what evidence permits this system to act here, how will we know when that permission is no longer justified, and who has the authority to withdraw it?