Uncertainty/Examples/Example 1
Jump to navigation
Jump to search
Example of Uncertainty
Let $R_1$ and $R_2$ be horseraces.
Let $R_1$ have $7$ runners:
- $3$ of which each have probability $\dfrac 1 6$ of winning
- $4$ of which each have probability $\dfrac 1 8$ of winning.
Let $R_2$ have $8$ runners:
- $2$ of which each have probability $\dfrac 1 4$ of winning
- $6$ of which each have probability $\dfrac 1 {12}$ of winning.
Then $R_1$ and $R_2$ have equal uncertainty.
Proof
Let $H_1$ and $H_2$ denote the uncertainty of $R_1$ and $R_2$ respectively.
Recall the definition of uncertainty:
- $\map H X = \ds -\sum_k p_k \lg p_k$
Thus:
\(\ds H_1\) | \(=\) | \(\ds -\paren {\sum_{k \mathop = 1}^3 \dfrac 1 6 \lg \dfrac 1 6 + \sum_{k \mathop = 1}^4 \dfrac 1 8 \lg \dfrac 1 8}\) | ||||||||||||
\(\ds \) | \(=\) | \(\ds -\dfrac 1 6 \paren {3 \lg \dfrac 1 6} - \dfrac 1 8 \paren {4 \lg \dfrac 1 8}\) | simplifying | |||||||||||
\(\ds \) | \(=\) | \(\ds -\dfrac 1 2 \lg \dfrac 1 6 - \dfrac 1 2 \lg \dfrac 1 8\) | simplifying | |||||||||||
\(\ds \) | \(=\) | \(\ds \dfrac 1 2 \paren {\lg 6 + \lg 8}\) | Logarithm of Reciprocal | |||||||||||
\(\ds \) | \(=\) | \(\ds \dfrac 1 2 \lg 48\) | Sum of Logarithms |
and:
\(\ds H_2\) | \(=\) | \(\ds -\paren {\sum_{k \mathop = 1}^2 \dfrac 1 4 \lg \dfrac 1 4 + \sum_{k \mathop = 1}^6 \dfrac 1 {12} \lg \dfrac 1 {12} }\) | ||||||||||||
\(\ds \) | \(=\) | \(\ds -\dfrac 1 4 \paren {2 \lg \dfrac 1 4} - \dfrac 1 {12} \paren {6 \lg \dfrac 1 {12} }\) | simplifying | |||||||||||
\(\ds \) | \(=\) | \(\ds -\dfrac 1 2 \lg \dfrac 1 4 - \dfrac 1 2 \lg \dfrac 1 {12}\) | simplifying | |||||||||||
\(\ds \) | \(=\) | \(\ds \dfrac 1 2 \paren {\lg 4 + \lg 12}\) | Logarithm of Reciprocal | |||||||||||
\(\ds \) | \(=\) | \(\ds \dfrac 1 2 \lg 48\) | Sum of Logarithms |
Hence the result.
$\blacksquare$
Sources
- 1988: Dominic Welsh: Codes and Cryptography ... (previous) ... (next): $\S 1$: Entropy = uncertainty = information: $1.1$ Uncertainty: Exercises $1.1$: $1.$