Ever wonder what happens to those amazing breakthroughs you hear about on the news, but never hear about again? Somehow, when they're finally released, the amazing qualities of, say, that new wonder drug, never seem to reduce the suffering the way most people hoped.

Look through the reports on the test results of those breakthroughs, and you'll frequently find one line that says p < 0.05. In other words, the tests indicate that the results reported on in the report had only a 5% chance of happening randomly.

If I flip a coin 20 times, and heads shows up 15 or more times (in other words, greater than 14 times), we can work out that there is roughly a 2.07% chance of that happening at random. Reporting on this, we'd note that p < 0.05, and use this to justify examining whether the coin is really fair.

That works great for events dealing with pure randomness, such as coins, but how do you update the probabilities for non-random factors? In other words, how do you take new knowledge into account as you go? This is where Bayes' theorem comes in. It's named after Thomas Bayes, who developed it in the mid-1700s, but the basic idea has been around for some time.

You should be familiar, of course, with the basic formula for determining the probability of a targeted outcome:



The following video describes the process of Bayes' theorem without going into any more mathematics than the above formula, using the example of an e-mail spam filter:



To get into the mathematical theorem itself, it's important to understand a few things. First, Bayes' theorem pays close attention to the differences between the event (an e-mail actually being spam or not, in the above video) and the test for that event (whether a given e-mail passes the spam test or not). It doesn't assume that the test is 100% reliable for the event.

BetterExplained.com's post An Intuitive (and Short) Explanation of Bayes’ Theorem takes you from this premise and a similar example, all the way up to the formula for Bayes' theorem. It's interesting to note that it's effectively the same as the classic probability formula above, but modified to account for new knowledge.

The following video uses another example, and is also simple to follow, but delves into the math as well as the process. Understanding the process first, and then seeing how the math falls into place helps make it clear:



The tree structure used in this video helps dramatize one clear point. Bayes' theorem allows you to see a particular result, and make an educated guess as to what chain of events led to that result.

The p < 0.05 approach simply says “We're at least 95% certain that these results didn't happen randomly.” The Bayes' theorem approach, on the other hand, says “Given these results, here are the possible causes in order of their likelihood.”

If I shuffle a standard 52-card deck, probability tells us that the odds of the top card being an Ace of Spades is 1/52. If I turn up the top card and show you that it's actually the 4 of Clubs, our knowledge not only chance the odds of the top card being the Ace of Spades to 0/52, but gives us enough certain data we can switch to employing logic. Having seen the 4 of Clubs on top and knowing that all the cards in the deck are different, I can logically conclude that the 26th card in the deck is NOT the 4 of Clubs.

We can switch from probability to logic in this manner because we've gone from randomness to certainty. What if I don't introduce certainty, however? What if I look at the top card without showing it to you, and only state that it's an Ace?

This is the strength of Bayes' theorem. It bridges the ground between probability with logic, by allowing you to update probabilities based on your current state of knowledge, not just randomness. That's really the most important point about Bayes' theorem.

There's much more to Bayes' theorem than I could convey in a short blog post. If you're interested in a more in-depth look, I suggest the YouTube video series Bayes' Theorem for Everyone. I think you'll find it surprisingly fascinating.