How Do You Know When You Have Reconstructed Enough?
There is no defensible number of interviews, incidents, or pages that proves an expert’s judgment has been preserved. The work is deep enough when the consequential decision families are represented by varied cases, important cues and exceptions have survived challenge, another informed person can use the material, and the remaining uncertainty is visible enough for the company to manage deliberately.
By Ken Ohyama, Founder · Published August 30, 2026 · Reviewed August 30, 2026
- expert judgment
- Critical Decision Method
- knowledge transfer
At a glance
Key takeaways
- Interview volume is a weak measure because different cases reveal different parts of expert work.
- The appropriate depth depends on consequence, variation, purpose, and the availability of another informed reader or successor.
- Contrasting cases and challenges to early explanations are more useful than repeatedly collecting familiar stories.
- A responsible stopping decision records what remains uncertain instead of implying that the person has been completely captured.
The box gets heavier while the question stays open
The interviews are complete. The transcripts are labeled. A binder has become two binders, then a searchable folder. Everyone can see that work happened. No one can yet answer the awkward question: if Mike leaves tomorrow, is the company ready?
Volume is comforting because it can be counted. Judgment is harder. The fourth account of a familiar pricing decision may add little; one unusual failure may reveal the threshold that makes the other accounts intelligible. Another ten hours can deepen the record or simply make it longer.
A disciplined project therefore needs a way to stop that does not depend on fatigue, budget exhaustion, or a promise that a certain number of conversations will somehow contain a career.
The research supports better questions, not a magic count
Applied Cognitive Task Analysis was developed as a streamlined approach because fuller cognitive task analysis can be resource intensive. Militello and Hutton describe three interview methods for identifying cognitive demands and translating them into training or design products. Their evaluation found the techniques usable and the resulting material useful, but it did not establish a universal number of interviews that makes a reconstruction complete.[Applied cognitive task analysis (ACTA): a practitioner's toolkit for understanding cognitive task demands]
In a neonatal intensive-care study, Critical Decision Method interviews elicited significantly more information than conventional interviews. Researchers used cues and examples from those incident accounts to build an early-sepsis assessment guide containing material absent from the literature available at the time. The result shows that interview method changes what becomes visible. It does not tell an executive team how many incidents are enough for a different domain.[Critical Decision Method: A Technique for Eliciting Concrete Assessment Indicators From the “Intuition” of NICU Nurses]
The honest conclusion is less convenient than a package count. Depth depends on the decisions, the range of conditions, the intended use, the distribution of expertise, and what would happen if something important were missed.
Research gem
Critical Decision Method interviews elicited significantly more information than conventional interviews from experienced NICU nurses and surfaced assessment cues absent from the literature then available.
Method note: The study concerned neonatal intensive care. It supports incident-centered elicitation, not a claim that CDM completely reconstructs expertise or transfers directly to every executive setting.
Start with the decisions that deserve the room
A company can spend months preserving detail that no future decision will need. Begin instead with consequence. Which decisions could materially affect a customer, cash, safety, people, or the direction of the business? Which still bend toward one person when conditions become unusual?
Group those decisions into families. A customer exception may look different from a supplier failure, yet both may turn on the same promise, risk threshold, or reading of trust. The family matters because a successor must recognize the underlying shape when the surface details change.
The first boundary is therefore selection. Leave routine material with the systems that already own it. Give the scarce transition window to the calls where history, cues, exceptions, and consequence meet.
Look for variation before looking for completion
A clean success story usually confirms what everyone already believes. Ask for the case that nearly failed, the time the normal answer was rejected, the decision that looked similar but turned out differently, and the occasion when the expert changed their mind late. Contrast makes the boundary easier to see.
Hinds, Patterson, and Pfeffer found that experts teaching an electronic-circuit task used more abstract and advanced statements and fewer concrete statements than beginners. Novices taught by beginners did better on the immediate task, while expert instruction helped more on a different task in the same domain. The research complicates a common assumption: expert abstraction can omit immediate steps and still carry patterns that matter for transfer.[Bothered by Abstraction: The Effect of Expertise on Knowledge Transfer and Subsequent Novice Performance]
That is why reconstruction needs both incident detail and the larger pattern the expert believes the incident represents. One without the other leaves either a pile of stories or a collection of principles no one can reliably apply.
Let someone else try to use what you preserved
The record may sound complete while its original author is in the room. Give a case to another informed person. Ask what they notice, what they would do next, what would make them pause, and which part of the record they could not interpret without Mike. The gaps become concrete quickly.
A successor is especially useful here, but validation does not require pretending every project includes a named heir. A peer, second-line leader, or subject-matter neighbor can challenge terminology, missing context, and assumed access. The objective is a record that can travel beyond the person who made it.
This is where reconstruction meets readiness. The Map can make selected decisions, cases, relationships, and thresholds visible. The Passage asks whether another person can carry them under changed conditions. One artifact may support both, but the second question requires performance rather than confidence in the document.
Skagway practitioner test
Five questions before the work closes
These are stopping disciplines, not a validated completeness score.
01Have the consequential decision families been covered?Scope follows consequence.
The record should represent the small number of decisions whose loss would matter materially, rather than attempt to preserve every responsibility attached to the person.
02Have contrasting cases changed the first explanation?Variation reveals boundaries.
Look for failures, exceptions, near misses, and similar-looking cases with different answers. If every case confirms the same neat rule, the inquiry may still be too shallow.
03Are cues, thresholds, and exceptions intelligible?The decision needs more than an outcome.
Another informed reader should be able to see what drew attention, what changed the interpretation, and where the usual answer stopped applying.
04Has another person tried to use the record?Transfer exposes hidden assumptions.
A peer or successor should work through a relevant case without the original expert silently supplying the missing context.
05Is the residual uncertainty visible?Blank spaces belong on the map.
Unobserved rare cases, disagreement, personal trust, and changing conditions should be recorded with review or escalation triggers rather than hidden behind a claim of completeness.
Stop with a ledger of what remains unknown
Some cases occur too rarely to observe. Memory will be incomplete. Two experts may disagree. A relationship may depend on trust that cannot be represented in a note. Strategy may change before the preserved judgment is ever used. These are reasons to mark uncertainty, not reasons to continue interviewing forever.
A responsible close names the covered decision families, the cases used, the people who challenged the record, and the dependencies that remain. It also names a review trigger: a changed strategy, a new authority boundary, a lost relationship, or a case that the existing map cannot explain.
The earlier article about asking an expert “why” explains why ordinary explanation often stops too soon. The harder discipline is knowing when a careful reconstruction has gone far enough—and leaving the next person a map with honest blank spaces.
Illustrative example
A retiring operating leader describes six supplier interruptions. The first four produce the same response. The fifth reveals that a promised allocation threshold changes when two customers share a downstream dependency. The sixth shows that the threshold no longer applies when regulatory exposure is involved. The project closes after a second leader can work through new variations and the remaining legal question has been assigned to counsel—not after reaching a predetermined interview count.
When Skagway is a fit
Skagway Succession is a U.S. executive-succession advisory that captures and transfers the tacit judgment of critical leaders. We are a fit when an organization needs a deliberate, evidence-led process for a critical executive, founder, technical expert, or operator. We are not a replacement for legal, tax, executive-search, compensation, fiduciary, or broad leadership-development advice.
Explore The MapGlossary
- Decision family
- A group of cases that share an underlying judgment pattern even when their surface details differ.
- Contrasting case
- A case selected because it resembles another case but produces a different interpretation or action.
- Residual uncertainty
- Important judgment, context, or dependency that remains incomplete, disputed, untested, or likely to change after the reconstruction closes.
- Stopping test
- A practical criterion used to decide whether additional elicitation is likely to change the usefulness of the record.
Sources & further reading
- Applied cognitive task analysis (ACTA): a practitioner's toolkit for understanding cognitive task demands (opens in a new tab) · Laura G. Militello and Robert J. B. Hutton · Ergonomics, 41(11), 1618–1641 · Taylor & Francis · 1998
- Critical Decision Method: A Technique for Eliciting Concrete Assessment Indicators From the “Intuition” of NICU Nurses (opens in a new tab) · Beth Crandall and Karen Getchell-Reiter · Advances in Nursing Science, 16(1), 42–51 · Lippincott · 1993
- Bothered by Abstraction: The Effect of Expertise on Knowledge Transfer and Subsequent Novice Performance (opens in a new tab) · Pamela J. Hinds, Michael Patterson, and Jeffrey Pfeffer · Journal of Applied Psychology, 86(6), 1232–1243 · American Psychological Association · 2001
This guide is founder-led analysis. Sources provide background and are not endorsements of Skagway Succession.
Continue the research
What took decades to learn
should not disappear in a day.
The road ahead should remember how the company came this far.
