Math for Real Life
The Answer That Hasn't Moved in 36 Years
— All my life people have told me that mathematics teaches you to think. And all my life I've nodded along. But let's be honest: I passed calculus at university, I forgot it within a year, and I can't remember a single occasion when it helped me choose a job or work out whether my doctor was lying to me. So what does it actually give me — beyond the right to say that line?
— Let me give you a problem instead of an answer. There's a disease; one person in a thousand has it. There's a test for it. The test gives a false alarm in 5% of cases — meaning that in a healthy person it comes back positive 5% of the time. You take it. You're positive. What's the probability that you have the disease?
— Well, if it's wrong 5% of the time, then I'm about 95% likely to have it.
— About two.
— Hold on. Say the conditions again, I think I misheard something.
— The conditions are exactly what I said. One sick person in a thousand, 5% false alarms. And your reaction is exactly the one everybody has: not to argue, but to ask again. At this point people don't think they've made a mistake — they think they've been given the wrong problem.
— All right, I'm not a doctor. Maybe I misread the wording — I'm not used to that kind of language.
— That's exactly what they asked doctors. 1978, Harvard Medical School: they caught people in the corridors — practising physicians, residents, students. 60 people. 27 of them answered "95%" — the same as you. 11 got it right.
— 1978 is half a century ago. Back then statistics in medicine was taught however it happened to be taught.
— In 2014 the same question was put again, to different people, in Boston clinics. 61 people. 27 said "95%". 14 got it right. Between the two measurements sit 36 years of medical education reform, and the distribution of answers barely shifted by a single percentage point.
— That's unpleasant to hear.
— Let me flag something straight away, so you don't walk out of here with more than the data supports: both samples are small, around 60 people each, and both come from one region. This isn't a national measurement, it's an illustration. But an illustration with a rare property — it reproduces.
— And why does everyone get it wrong in the same way? If it were just carelessness, the errors would scatter in different directions.
— That's the whole point. The error isn't random. You can predict it in advance knowing nothing but the wording of the problem — and not one person inside it feels that they've made a mistake. The brain isn't computing a probability. It's assessing similarity: the test for the disease showed the disease, therefore disease. And it substitutes that answer for the calculation silently, without announcing the swap.
— So it isn't that I calculated wrong. I didn't calculate at all.
— You answered an easier question. The one that sounds almost the same. Let me redo it out loud — with the very same numbers. Take a thousand people. One of them is sick, and the test will almost certainly catch them. That leaves 999 healthy people, and 5% of them is roughly 50 people whose test will also come back positive.
— So there are 51 positives, and one sick person among them. That's... 2%.
— Yes.
— But you gave me different numbers before!
— The same ones. I didn't tell you a single new fact. I rearranged them so you wouldn't have to divide fractions in your head.
— And does that work on anyone?
— Not on anyone, but reliably and on a great many people — so reliably that an entire position has grown out of it. That's Gigerenzer's position, and it's fairly radical: he says it isn't the human that's broken, it's the question. That what we call an error is often optimal — with little data and not enough time, a simple rule beats a complex model. Doctors in his picture aren't defective. What's defective is the format the numbers are handed to them in.
— Then why is the brain built that way at all? From a survival point of view that's a straight manufacturing defect.
— There's an explanation, but I'm obliged to be honest about what kind of explanation it is. Haselton and Buss, and separately Nesse, suggest thinking about it as a smoke alarm: it screams at toast, and nobody fixes it, because the cost of a false alarm is a ruined breakfast and the cost of a miss is the house. A fast wrong answer is cheaper than a slow right one. But that's an explanatory frame from evolutionary psychology, reconstructed after the fact. That's how it's explained — not "that's how it works".
— And why do you keep hedging? You're supposed to be arguing your own side here.
— Because you'd ask anyway later on, and I'd rather say it myself. The psychology all of this comes from has been put to the test, and it came through unevenly. In 2015, 100 studies were repeated. There was a significant result in 97 of the originals and in 36 of the replications.
— And you're telling me this voluntarily.
— In 2017 Kahneman himself publicly agreed with the critics and admitted that he had trusted underpowered studies too much, and that some of the effects in his own book cannot be as large as he wrote. The picture is uneven, though: anchoring survived a mass replication across 36 laboratories, while loss aversion was declared unfounded in 2018 — and immediately drew a detailed rebuttal. The argument is open.
— Then what in all this am I supposed to believe?
— What reproduces. That people ignore the base rate — that has held since 1978 and repeated in 2014 on a different sample. Whereas the list of 200 cognitive biases in the popular books is a rough draft, not the periodic table. And the fact that you just winced is the right reaction. It's going to get worse, because we're going to get to examples I told students myself for 20 years and that turned out to be invented.