The Bayesian Trap - Veritasium

The other day I watched a very cool video produced by Veritasium that highlighted a problem with false-positives in medical testing and how easy it is to over-simply test results.  The video is called The Bayesian Trap.

I highly encourage you to watch it yourself but to fill you in, the video focuses on a hypothetical situation in which you've gone to the doctor to determine what is wrong with you.  The doctor runs a test and discovers you tested positive for a pretty horrible disease that effects 1 in 1,000 people.  The test has a 99% accuracy rate.  So what are the odds you have this disease?  99%, right?  That's what common sense tells us, but in this case common sense is very wrong.

The video walks you through the steps to determine the correct odds you have the disease.  You may have tested positive, but a test with a 99% accuracy rate has a 1% false-positive rate.  This means that 1% of individuals testing positive for the disease will in fact not have the disease!  Take 1,000 people.  Of those 1,000, one will have the disease, but 1% will falsely test positive.  Out of the 1,000, 10 will test falsely test positive in addition to the person who correctly tests positive.  There are now 11 people testing positive, only one of which actually has the disease.  Therefore your odds of having the disease after testing positive are 1 in 11 or 9.1%.  That's it!

This is why medical professionals are very concerned at the number of false positives out there in which people test positive for a disease and are then treated for a disease they don't have.  In many cases, treatment may come with serious side effects.  Thus it is important to avoid as many false positives as possible.  One way to remove false positives is to run multiple tests.  In the example above, if the test is ran twice, the odds of a false positive drop dramatically and your odds of correctly testing positive (incorrectly testing positive twice) jump to around 91%.

If you haven't already seen this video, please watch it.  It's a great example of how common sense leads us to an often very incorrect assumption.