Khinchin's Law

Theorem
Let $P$ be a population.

Let $P$ have mean $\mu$ and finite variance.

Let $\sequence {X_n}_{n \mathop \ge 1}$ be a sequence of random variables forming a random sample from $P$.

Let:


 * $\displaystyle {\overline X}_n = \frac 1 n \sum_{i \mathop = 1}^n X_i$

Then:


 * ${\overline X}_n \xrightarrow p \mu$

where $\xrightarrow p$ denotes convergence in probability.

Proof
Let $\sigma$ be the standard deviation of $P$.

By the definition of convergence in probability, we aim to show that:


 * $\displaystyle \lim_{n \mathop \to \infty} \map \Pr {\size { {\overline X}_n - \mu} < \varepsilon} = 1$

for all real $\varepsilon > 0$.

Let $\varepsilon > 0$ be a real number.

By Variance of Sample Mean:


 * $\var {{\overline X}_n} = \dfrac {\sigma^2} n$

By Chebyshev's Inequality, we have for real $k > 0$:


 * $\map \Pr {\size {{\overline X}_n - \mu} \ge \dfrac {k \sigma} {\sqrt n}} \le \dfrac 1 {k^2}$

As $\sigma > 0$ and $n > 0$, we can set:


 * $k = \dfrac {\sqrt n} {\sigma} \varepsilon$

This gives:


 * $\map \Pr {\size {{\overline X}_n - \mu} \ge \varepsilon} \le \dfrac {\sigma^2} {n \varepsilon^2}$

We therefore have:

So:


 * $1 - \dfrac {\sigma^2} {n \varepsilon^2} \le \map \Pr {\size { {\overline X}_n - \mu} < \varepsilon} \le 1$

We have:


 * $\displaystyle \lim_{n \mathop \to \infty} \paren {1 - \dfrac {\sigma^2} {n \varepsilon^2} } = 1$

and:


 * $\displaystyle \lim_{n \mathop \to \infty} 1 = 1$

So by the Squeeze Theorem:


 * $\displaystyle \lim_{n \mathop \to \infty} \map \Pr {\size { {\overline X}_n - \mu} < \varepsilon} = 1$

for all real $\varepsilon > 0$.

Also see

 * Strong Law of Large Numbers