Variance as Expectation of Square minus Square of Expectation

From ProofWiki
Jump to navigation Jump to search


Theorem

Let $X$ be a random variable.

Then the variance of $X$ can be expressed as:

$\var X = \expect {X^2} - \paren {\expect X}^2$


That is, it is the expectation of the square of $X$ minus the square of the expectation of $X$.


Proof

Discrete Random Variable

We let $\mu = \expect X$, and take the expression for variance:

$\var X := \displaystyle \sum_{x \mathop \in \Img X} \paren {x - \mu}^2 \Pr \paren {X = x}$

Then:

\(\displaystyle \var X\) \(=\) \(\displaystyle \sum_x \paren {x^2 - 2 \mu x + \mu^2} \Pr \paren {X = x}\)
\(\displaystyle \) \(=\) \(\displaystyle \sum_x x^2 \Pr \paren {X = x} - 2 \mu \sum_x x \Pr \paren {X = x} + \mu^2 \sum_x \Pr \left({X = x}\right)\)
\(\displaystyle \) \(=\) \(\displaystyle \sum_x x^2 \Pr \paren {X = x} - 2 \mu \sum_x x \Pr \left({X = x}\right) + \mu^2\) Definition of Probability Mass Function: $\displaystyle \sum_x \Pr \paren {X = x} = 1$
\(\displaystyle \) \(=\) \(\displaystyle \sum_x x^2 \Pr \paren {X = x} - 2 \mu^2 + \mu^2\) Definition of Expectation: $\displaystyle \sum_x x \Pr \paren {X = x} = \mu$
\(\displaystyle \) \(=\) \(\displaystyle \sum_x x^2 \Pr \paren {X = x} - \mu^2\)

Hence the result, from $\mu = \expect X$.

$\blacksquare$


Continuous Random Variable

Let $\mu = \expect X$.

Let $X$ have probability density function $f_X$.


As $f_X$ is a probability density function:

$\displaystyle \int_{-\infty}^\infty f_X \paren x \rd x = \Pr \paren {-\infty < X < \infty} = 1$


Then:

\(\displaystyle \var X\) \(=\) \(\displaystyle \expect {\paren {X - \mu}^2}\) Definition of Variance of Continuous Random Variable
\(\displaystyle \) \(=\) \(\displaystyle \int_{-\infty}^\infty \paren {X - \mu}^2 f_X \paren x \rd x\) Definition of Expectation of Continuous Random Variable
\(\displaystyle \) \(=\) \(\displaystyle \int_{-\infty}^\infty \paren {x^2 - 2 \mu x + \mu^2} f_X \paren x \rd x\)
\(\displaystyle \) \(=\) \(\displaystyle \int_{-\infty}^\infty x^2 f_X \paren x \rd x - 2 \mu \int_{-\infty}^\infty x f_X \paren x \rd x + \mu^2 \int_{-\infty}^\infty f_X \paren x \rd x\)
\(\displaystyle \) \(=\) \(\displaystyle \expect {X^2} - 2 \mu^2 + \mu^2\) Definition of Expectation of Continuous Random Variable
\(\displaystyle \) \(=\) \(\displaystyle \expect {X^2} - \mu^2\)
\(\displaystyle \) \(=\) \(\displaystyle \expect {X^2} - \paren {\expect X}^2\)

$\blacksquare$


Comment

This is a significantly more convenient way of defining the variance than the first-principles version. In particular, it is far easier to program a computer to calculate this (you don't need to maintain a record of all the divergences). Therefore, this is by far the more usually encountered of the definitions for variance.


Also see


Sources