Conditional probability, simply explained

What is the single most important concept in statistics?

Conditional probability. Without a way to account for prior information in our models, statistical models would be useless.

Let's see what conditional probability simply means!

If $\textstyle A$ and $\textstyle B$ are two events, they are not necessarily independent of each other. This means that the occurrence of one can give information about the other.

When performing statistical modeling, this is frequently the case. To illustrate, we will take a look at spam filters!

Suppose that you have 100 emails in your inbox. 40 is spam. 60 is not.

Based only on this information, if you receive a random letter, the best guess is a 40% chance of it being spam. This is not good enough for effective spam detection. Let's look at some additional information!

What about taking a look at the mail and checking for the word "offer"? It turns out that among the 48 mails containing this word, 36 are spam! Here, we can find this by simply counting.

So, let's formulate this in the language of probability! Based on this sample, if you know that a mail contains the word "offer", there is a 75% chance of being spam. In mathematical words, we have

$$P(\text{spam } | \text{ contains the word "offer"}) = \frac{36}{48} = 0.75.$$

Knowing this can help you make a better decision when filtering out unwanted spam mail.

Essentially, conditional probability restricts the event space to the observed event. It allows you to refine your statistical models upon gaining new information.

However, obtaining it is not always as simple. Often, you can't just simply count all the cases. For these situations, the Bayes formula provides a tractable way to update our beliefs about the world, given new information. (I recently wrote a blog post on this, check it out!)