Eight decision traps and how to keep a decision journal

Six months after opening a second branch, sales are half the plan. The owner reopens the approval file and finds a monthly revenue figure on the first line from the first meeting, and nobody remembers where it came from. The furniture shop in this story is hypothetical, but the question is real. How do you find out which trap you are in while you decide, not after the money is gone?

What are the eight traps of Hammond, Keeney and Raiffa?

The opening paragraph of Hammond, Keeney and Raiffa (1998) notes that bad decisions can often be traced to how they were made, but sometimes the fault lies in the mind of the decision maker, because the way the brain works can sabotage our choices (Hammond et al., 1998). The table below paraphrases their descriptions. The last column holds the question I would put on a journal page to target each trap.

The site library merges the three estimating traps (overconfidence, prudence, recallability) into one node, so its diagram shows six nodes rather than eight. The post has only the public summary of the original article, so each description above stays at one line (Hammond et al., 1998).

Is knowing the names enough to avoid the traps?

Not necessarily. The public summary of Hammond and colleagues says the best way to avoid all the traps is awareness, because to be forewarned is to be forearmed (Hammond et al., 1998). The chapter by Soll, Milkman and Payne (2015) is less comfortable.

People have a blind spot for their own biases, although they often spot the same biases in others. The chapter says it knows of no study that asks this directly, but advice to be careful when tired or moody is probably harmless, and the benefit may be small (Soll et al., 2015, p. 930). Statistical principles learned in the lab tend to fade within weeks when applied to new domains, and the most effective training is on the very kind of decision you are about to make, possibly repeatedly (Soll et al., 2015, p. 931).

Calibration also needs feedback. Weather forecasters are well calibrated within their field provided they get training, timely feedback and suitable incentives, yet away from their field they are as poorly calibrated as the rest of us (Soll et al., 2015, p. 931). According to the chapter's account of Ben-David, Graham and Harvey (2013), chief financial officers forecast yearly S&P 500 returns over nine years, and their "80% sure" ranges contained the true result only 33% of the time (Soll et al., 2015, p. 933).

Owners seldom get timely feedback. Opening a branch, changing a supplier or borrowing to expand happens a few times over many years, and by the time the result arrives, memory of the original reasons has shifted. A page that records reasons and predictions in advance is the cheapest way to get something to compare against. That is why I propose a journal.

Which three fronts should debiasing cover?

The public summary of "Outsmart your own biases" sorts techniques under three ways of widening your view: possible outcomes, objectives and options (Soll et al., 2015). The library uses the same arrangement in Debiasing on three fronts.

For the future front, the chapter offers two techniques. One is to ask separately for the 10th, 50th and 90th percentiles instead of a single range, which raised the hit rate by about 20 percentage points because the ranges were wider and better centred (Soll et al., 2015, p. 933). The other is "prospective hindsight" (the library's premortem): suppose the project has already failed, then explain why. The chapter notes that people find more routes to failure when looking back on one, even an imagined one, and says plainly that more research is needed (Soll et al., 2015, p. 932).

What goes into a seven-line decision journal?

One page, filled in at two moments. The template below is my own proposal, assembled from the techniques above, and untested.

DECISION JOURNAL (lines 1 to 6 before deciding)
1. What is the decision, who decides, and by what date?
2. Write the question two ways (for example "should we open?" and "what is the best use of this capital?").
3. At least three options, one of them "keep things as they are" with its price. Strike every sunk cost from the reasons.
4. The deciding figure: low, middle and high estimates (10th, 50th and 90th percentiles), plus the base rate for this kind of project. Write them before seeing anyone else's number.
5. Suppose this has failed in 12 months: three likely causes.
6. One thing that would change my mind, and one person I will ask to challenge me.
7. Review date. Actual result, gap against the estimate, lesson (filled in later).

Each line targets one or more traps in the first table. Line 2 targets framing, line 3 status quo and sunk cost, line 4 anchoring, overconfidence, prudence and recallability, and line 6 confirming evidence. Line 5, the look back from an imagined failure, targets excessive optimism. Line 7 is the feedback. For infrequent decisions, Soll and colleagues write that cognitive strategies for identifying objectives, generating a broad range of alternatives and seeking disconfirming evidence are likely to yield a high return (Soll et al., 2015, p. 943). The seven lines are my way of making that sentence concrete.

A hypothetical example: a furniture shop opens a second branch

Every figure below is invented by me for illustration. The shop does not exist.

The owner plans to lease a second site, with a deposit and fit-out of VND 2 billion. In the first meeting the manager says the existing branch sells VND 150 million a month, so the new one will do "no less than 150". That is the anchor. The owner fills in the journal:

The middle figure of 100 sits well below the anchor of 150. The journal does not forbid the second branch. It makes people sit with the number 100 before the deposit is signed.

The bigger value comes from repetition. Suppose that after ten estimates only 5 actual results fall between the low and high figures. A 10th-to-90th percentile range should contain about 8 in 10. If the estimates were well calibrated and the ranges independent, a result of 5 or fewer would turn up only about 3 per cent of the time (my own calculation from the binomial distribution). Your ranges are probably too narrow. Next time, lower the low and raise the high.

When does a journal fit, and when not?

It fits when the decision is large, hard to reverse and rarely met again: opening a branch, borrowing to expand, changing a main supplier. It does not fit decisions you make every week. Soll and colleagues write that for repeated decisions, linear models, checklists and consistent policies can dramatically improve accuracy (Soll et al., 2015, p. 943).

Three limits. None of the sources used here measures the effect of a journal; the post assembles techniques whose evidence is separate. A journal is only worth anything when it is filled in before the result is known. And ten estimates is a small sample, so one miss proves nothing.

To look back after a decision, read Gibbs's reflective cycle for managers. To replace one figure with a few scenarios, read scenario planning for business owners.

Going deeper: what the post rests on, and what it does not

Hammond, Keeney and Raiffa (1998) sits behind a paywall. The post uses the public summary and the opening paragraph. The names and one-line descriptions of the eight traps come from there, and the examples and remedies in the body of the article have not been used (Hammond et al., 1998). For "Outsmart your own biases" (2015) too, only the summary is used, and only for the three-front arrangement (Soll et al., 2015).

The chapter "A user's guide to debiasing" is used from page 924 to 944, in an uncorrected-proofs copy posted by one of its authors. Page numbers follow that copy, and the end of the chapter has not been used. The smaller studies inside it are known only through the chapter's account of them (Soll et al., 2015).

On criticism: the public summary gives no measurements or validating studies for the eight traps. Setting the two sources against each other is my framing, not the chapter's. So treat the journal as a small experiment inside your own business: record ten estimates, count how often your ranges hit, then adjust.