Bernoulli's Theorem

Theorem
Let the probability of the occurrence of an event be $p$.

Let $n$ independent trials be made, with $k$ successes.

Then:
 * $\ds \lim_{n \mathop \to \infty} \frac k n = p$

Proof
Let the random variable $k$ have the binomial distribution with parameters $n$ and $p$, that is:
 * $k \sim \Binomial n p$

where $k$ denotes the number of successes of the $n$ independent trials of the event with probability $p$.

From Expectation of Binomial Distribution:
 * $\expect k = n p \leadsto \dfrac 1 n \expect k = p$

Expectation is Linear gives:
 * $ \expect {\dfrac k n} = p =: \mu$

Similarly, from Variance of Binomial Distribution:
 * $\var k = n p \paren {1 - p} \leadsto \dfrac 1 {n^2} \var k = \dfrac {p \paren {1 - p} } n$

From Variance of Linear Combination of Random Variables:
 * $\var {\dfrac k n} = \dfrac {p \paren {1 - p} } n =: \sigma^2$

By applying the Bienaymé-Chebyshev Inequality to $\dfrac k n$, we have for any $l > 0$:
 * $\map \Pr {\size {\dfrac k m - \mu} \ge l \sigma} \le \dfrac 1 {l^2}$

Now, let $\epsilon > 0$ and choose $l = \dfrac \epsilon \sigma$, to get:
 * $\map \Pr {\size {\dfrac k m - \mu} \ge \dfrac \epsilon \sigma \cdot \sigma} \le \dfrac {\sigma^2} {\epsilon^2}$

Simplifying and plugging in the values of $\mu$ and $\sigma^2$ defined above yields:
 * $\map \Pr {\size {\dfrac k n - p} \ge \epsilon} \le \dfrac {p \paren {1 - p} } {n \epsilon^2}$

Scaling both sides by $-1$ and adding $1$ to both sides yields:
 * $1 - \map \Pr {\size {\dfrac k n - p} \ge \epsilon} \ge 1 - \dfrac {p \paren {1 - p} } {n \epsilon^2}$

Applying Union of Event with Complement is Certainty to the :
 * $\map \Pr {\size {\dfrac k n - p} \le \epsilon} \ge 1 - \dfrac {p \paren {1 - p} } {n\epsilon^2}$

Taking the limit as $n$ approaches infinity on both sides, we have:
 * $\ds \lim_{n \mathop \to \infty} \map \Pr {\size {\frac k n - p} < \epsilon} = 1$

Also see

 * Khinchin's Law, otherwise known as the Weak Law of Large Numbers
 * Kolmogorov's Law, otherwise known as the Strong Law of Large Numbers