Some kinds of probability puzzles are particularly difficult:
1% of women at age forty who participate in routine screening have breast cancer. 80% of women with breast cancer will get positive mammographies. 9.6% of women without breast cancer will also get positive mammographies. A woman in this age group had a positive mammography in a routine screening. What is the probability that she actually has breast cancer?
Over 80% of doctors get this wrong. They tend to estimate that the woman with a positive mammogram has an 70-80% of really having cancer. The real answer is about 7.8%:
Out of 10,000 women, 100 have breast cancer; 80 of those 100 have positive mammographies. From the same 10,000 women, 9,900 will not have breast cancer and of those 9,900 women, 950 will also get positive mammographies. This makes the total number of women with positive mammographies 950+80 or 1,030. Of those 1,030 women with positive mammographies, 80 will have cancer. Expressed as a proportion, this is 80/1,030 or 0.07767 or 7.8%.
Sure, the fraction of women with false positives is much lower than those with true positives, but the percentage of women without cancer is so high that the raw numbers of cancer-free women to which we apply the 9.6% false-positive rate swamp the low rate. Similarly the percentage of women with cancer is so low that the high 80% true-positive rate is undermined by the low raw numbers of cancer-suffering women.
H/t Slate Star Codex archives mentioned in an open thread this weekend at Astral Codex Ten.
I used to see the same thing from suicide screenings. The professionals doing the evaluations did not understand what the numbers meant.
ReplyDeletePeople who don't know what Type I and Type II errors are (at least the concept, even if they don't know the name) should not be interpreting tests.
Casinos do well because most people can't even do the first calculation well; once you have to worry about the second-order effects of false positives and negatives, you've lost most of humanity.
ReplyDeleteInterestingly you get a lot closer if you stick with the base percentage: "Out of 10,000 women, one percent will have breast cancer." One percent is much closer to seven-point-eight percent than it is to eighty percent. Depending on how much the subsequent medical intervention entail substantial health risks to the patient, at some point it might be wiser to forgo being tested.
Several months ago, there was a multinational survey in which people were asked 'what is the chance of dying from Covid in your country?' The answers were bizarrely high, 20-30% range....oddly, even in Germany, where the Covid death rate was at the time of the survey very low.
ReplyDeleteThe guesses were also higher for young people than for older people.
At the same time, we didn't do well with the rough estimate of how many thousands of nursing home residents would die if we released convalescing COVID patients into them--the answer being 11,000 in New York. Making contact between an unusually dangerous group of contagious patients and an unusually vulnerable group of nursing home patients was like putting fire to gasoline, but we were so focused on marginally changing very, very small risks to the rest of the population that we were willing to put 20MM people out of work.
ReplyDelete