What is the expected value really?

Tivadar Danka small portrait Tivadar Danka
expected value of a discrete random variable

Understanding math will make you a better engineer.

So, I am writing the best and most comprehensive book about it.

Expected value is one of the most fundamental concepts in probability theory and machine learning.

Have you ever wondered what it really means and where does it come from? The formula doesn't tell the entire story right away.

Let's unravel what is behind the scenes!

E[x]=i=1nxiP(X=xi)\mathbb{E}[x] = \sum_{i=1}^{n} x_i P(X = x_i)

First, let's take a look at a simple example. Suppose that we are playing a game. You toss a coin, and

  • if it comes up heads, you win $1,
  • but if it is tails, you lose $2.

Should you even play this game with me? 🤔

After 𝑛 rounds, your earnings can be calculated by the number of heads times 1 minus the number of tails times 2. If we divide total earnings by n\textstyle n, we obtain the average earnings per round. What happens if n\textstyle n approaches infinity?

average earnings=1×#headsn2×#tailsn \text{average earnings} = 1 \times \frac{\# \text{heads}}{n} - 2 \times \frac{\# \text{tails}}{n}

As you have probably guessed, the number of heads divided by the number of tosses will converge to the probability of a single toss being heads. In our case, this is 1/2\textstyle 1/2. (Similarly, tails/tosses also converges to 1/2\textstyle 1/2.)

#headsnP(heads)=12(n) \frac{\# \text{heads}}{n} \to P(\text{heads}) = \frac{1}{2} \quad (n \to \infty)

So, your average earnings per round are 1/2\textstyle -1/2. This is the expected value, calculated by

average earnings=P(heads)2P(tails)=12.\begin{align*} \text{average earnings} &= P(\text{heads}) - 2 P(\text{tails}) \\ &= - \frac{1}{2}. \end{align*}

You definitely shouldn't play this game. How can we calculate the expected value for a general case?

Suppose that, similarly to the previous example, the outcome of your experiments can be quantified. (Like throwing a dice or making a bet at the poker table.) The expected value is just the average outcome you have per experiment when you let it run infinitely.

expected value=possible outcomes(value of outcome)×P(outcome)\text{expected value} = \sum_{\text{possible outcomes}} (\text{value of outcome}) \times P(\text{outcome})

The formula above is simply the expected value in English. If we formally denote the variable describing the outcome of the experiment with X\textstyle{X} and its possible values with xix_i we get back the formula from the beginning:

E[x]=i=1nxiP(X=xi).\mathbb{E}[x] = \sum_{i=1}^{n} x_i P(X = x_i).

It looks much easier now, isn't it?

Having a deep understanding of math will make you a better engineer.

I want to help you with this, so I am writing a comprehensive book that takes you from high school math to the advanced stuff.
Join me on this journey and let's do this together!