AI Can Reduce Key-Person Risk—and Create a New One

A company has one person who knows how to answer the difficult questions. It trains an internal assistant on approved records, past cases, and Mike’s recurring guidance. Soon more employees can find an answer without waiting for him. The queue shrinks. The company feels less dependent.

Then the model changes. A source system falls out of date. The vendor alters a feature. A rare customer case produces a smooth answer that Mike would have stopped. Everyone knows how to use the system; only two people still know enough to argue with it.

The old dependency was easy to recognize because it had a name. The new one is spread across technology, records, contracts, workflows, and human attention. It can be more resilient than the old arrangement and harder to see clearly.

In the field study of 5,179 customer-support agents, Brynjolfsson, Li, and Raymond found a 14 percent average productivity improvement from a generative-AI assistant and a 34 percent improvement among novice and lower-skilled workers. They report suggestive evidence that the system disseminated practices associated with stronger performers.[Generative AI at Work]

That finding matters for succession. Some work that once waited for an experienced person may become available through a tool. Employees can encounter stronger responses earlier, handle more cases, and draw on a record larger than any one colleague could remember at once.

The existing article AI Can Make Expertise Less Scarce develops that evidence in depth. The present question begins one step later: after capability is distributed, what does the company now depend on to keep the answer reliable?

The model may depend on access to current company records. Those records may depend on one team that understands their provenance. The workflow may depend on a vendor whose terms, model behavior, or service availability the company does not control. The final answer may depend on a small group who know which customer promise or safety boundary is absent from the training material.

The technical system may be far less fragile than the original arrangement around Mike. Even so, “people no longer need to call him” remains an incomplete measure of continuity.

A second route should exist for the capability, not merely for the software. If the model is unavailable, wrong, or no longer suited to the case, who can recognize that condition and carry the decision another way?

Parasuraman and Manzey’s review found automation complacency and automation bias among both inexperienced and expert users. Imperfect decision aids could induce omission and commission errors, and simple training or instructions did not eliminate the effects described in the reviewed evidence.[Complacency and Bias in Human Use of Automation: An Attentional Integration]

Lyell and Coiera’s systematic review screened 890 papers and included 40 studies. It found automation bias in single-task settings as well as multitasking settings, particularly where verification was cognitively difficult. The literature was fragmented, and relatively few studies reported statistical significance against a control condition.[Automation Bias and Verification Complexity: A Systematic Review]

Modern survey evidence adds a current but more limited signal. In 319 knowledge workers, higher confidence in generative AI was associated with less self-reported critical-thinking effort, while critical thinking shifted toward verification, integration, and stewardship. The study was correlational and cannot show that AI caused capability loss.[The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers]

Start with a consequential decision the AI now supports. Trace the source records, the model or service, the surrounding workflow, the person who owns the context, the person expected to verify the answer, and the authority that decides whether to proceed. At each point, ask what fails when that element is absent or wrong.

Only after that map exists do cue recognition, decision thresholds, exceptions, and scenario rehearsal become useful. Which signal should make a reviewer distrust an otherwise polished answer? Where must the system escalate? What contrasting case would reveal that the model has learned the recurring pattern and missed the dangerous boundary?

A Human in the Loop Is Not a Control explains why nominal review is too weak. The company needs an evaluator who has enough current, practiced capability to disagree—and a route through which that disagreement can change the decision.

A board can leave prompt inspection and technical testing to the appropriate specialists while still demanding a credible account of the consequential capabilities moving into AI: what the system is allowed to influence, what evidence supports that boundary, who can override it, and what happens if the tool or evaluator is unavailable. NIST’s AI Risk Management Framework likewise treats AI risk as an organizational lifecycle responsibility rather than a property of the model alone.[Artificial Intelligence Risk Management Framework (AI RMF 1.0)]

The apprenticeship question belongs in the same review. If the system handles the cases that once formed future evaluators, the company must decide how people will continue to build and demonstrate the unaided capability needed at the edge.

The Atlas can examine concentrated decisions and second routes across people and organizational systems. Model audits, AI-safety certification, cybersecurity, legal advice, data governance, and technical assurance remain with the appropriate specialists. The older question beneath the new technology still belongs here: if the familiar answer fails, who or what can make the call?