Noise by Daniel Kahneman, Olivier Sibony, and Cass R. Sunstein

Part 1: Finding noise

• Needless variability occurs between identical cases judged by different people, in the criminal justice system (even based on time of day or whether the local sports team won or lost), and in the insurance industry for making insurance claims. This is noise that causes needless injustice and financial costs

  Part 2: Your mind is a measuring instrument

• “Judgment is measurement in which the instrument is a human mind. Implicit in measurement is the goal of accuracy—to approach truth and minimize error.”
• Measurement is the assigning of a value produced by an instrument to some object or process.
• When predicting the likelihood of some event, there is variability produced by what we decide to attend to, the informality of the judgment process (e.g., not having a plan of what to attend to), and the conversion of the overall judgment to a number (the likelihood)
• If the noise (standard deviation) or bias (average) of an error are reduced by equal amounts, they have equal effect in reducing error. But our intuition tells us that centering our error distribution closer to zero is better (i.e., reducing bias). This is not true, reducing the spread of the distribution is just as effective.
• Level noise measures variability in the average level of judgments by different judges (between judge variability)
• Pattern noise is variability in judges’ responses to particular cases (within judge + between case variability)
• Occasion noise is variability due to randomness (within judge variability)
• Wisdom of crowds effect: the average answer of a large number of people is likely to be close to the truth (if the questions are relatively simple and don’t require expertise)
• The wisdom of crowds effect tends to increase accuracy because the average smears out the noise inherent in the individual judgments
• This also works with the crowd within. When we ask ourselves the same question multiple times, we tend to come a little closer to the truth. The larger the time that passes between judgments, the better the effect (the two members from the crowd within are further apart)
• You are not the same person at all times. When our mood varies, our cognitive machinery changes along with it.
• A study of nearly seven hundred thousand primary care visits showed that doctors become more likely to prescribe opioids as the end of the day nears. Under time-pressure, we lean towards quick-fix solutions.
• Wisdom of the crowds has a necessary precondition: independence. If judgements are not independent, they risk being swayed by other judgments.
• “If group members are listening to one another, they will shift in the direction of the dominant tendency, rendering the group more unified, more confident, and more extreme. And if people care about their reputation within the group, they will shift in the direction of the dominant tendency, producing polarization.”

  Part 3: Noise in predictive judgments

• Why do decision makers neglect objective ignorance (how much information they lack)? “When they listen to their gut, decision makers hear the internal signal and feel the emotional reward it brings. This internal signal that a good judgment has been reached is the voice of confidence, of ‘knowing without knowing why.’”
• Decision makers value internal coherence and satisfaction of reaching closure on a judgment more than actually getting the judgment correct.
• Causal thinking lures us into a false certainty about why things turned out the way they do. When a situation unfurls, and every step is plausible, we tell ourselves “of course it turned out this way” and model our future predictions accordingly. What we fail to realize is how differently things could have turned out if another plausible explanation occurred at an intermediate stage.

  Part 4: How Noise Happens

• “We think we base our opinions on evidence, but the evidence we consider and our interpretation of it are likely to be distorted, at least to some extent, to our initial snap judgment. As a result, we maintain the coherence of the overall story that has emerged in our mind.”
• Judgment is an operation that assigns a value on a scale to a subjective impression
  • Subjective impression…when we sense something, our conscious awareness of it is an impression—an impression created by sense organs transforming one form of energy into electrical energy sent to the neurons in our brains, leaving impressions in our minds. Like a handprint left in a memory foam mattress, we confuse the handprint for the hand.
• The anchor effect influences sequences of judgments. When predicting something quantitative, like a dollar penalty for a crime, dollar estimates of the first case presented will influence downstream estimates, since it will serve as a reference value. If different people receive different cases first, this creates noise.
  • To reduce this, asking instead for a ranking and then establishing a cost to be applied based on the relative ranking reduces this level noise (the noise between judges due to different anchors)
• “Subjective confidence in one’s judgment by no means guarantees accuracy. Moreover, the suppression of alternative interpretations…could induce the….illusion of agreement. If people cannot imagine possible alternatives to their conclusions, they will naturally assume that other observers must reach the same conclusions, too.”
• Bias is a compelling figure that demands our attention, noise is the background which we pay no attention to

  Part 5: Improving judgments

• “Noise is an invisible enemy, and preventing the assault of an invisible enemy can yield only an invisible victory.”