Total Probability Theorem/Conditional Probabilities

Theorem

Let $\struct {\Omega, \Sigma, \Pr}$ be a probability space.

Let $\set {B_1, B_2, \ldots}$ be a partition of $\Omega$ such that $\forall i: \map \Pr {B_i} > 0$.

Let $C \in \Sigma$ be an event independent to any of the $B_i$.

Let $\map \Pr C > 0$.

Then:

$\ds \forall A \in \Sigma: \condprob A C = \sum_i \condprob A {C \cap B_i} \, \map \Pr {B_i}$

Proof

First define $Q_C := \condprob {\, \cdot} C$.

Then, from Conditional Probability Defines Probability Space, $\struct {\Omega, \Sigma, Q_C}$ is a probability space.

Moreover:

\(\ds \forall i: \, \)

\(\ds \map {Q_C} {B_i}\)

\(=\)

\(\ds \map \Pr {B_i \mid C}\)

\(\ds \)

\(=\)

\(\ds \map \Pr {B_i}\)

$C$ and $B_i$ are independent

\(\ds \)

\(>\)

\(\ds 0\)

Therefore the Total Probability Theorem also holds true.

Hence we have:

\(\ds \map {Q_C} A\)

\(=\)

\(\ds \sum_i \map {Q_C} {A \mid B_i} \, \map {Q_C} {B_i}\)

Total Probability Theorem

\(\ds \)

\(=\)

\(\ds \sum_i \frac {\map {Q_C} {A \cap B_i} } {\map {Q_C} {B_i} } \, \map {Q_C} {B_i}\)

Definition of Conditional Probability

\(\ds \)

\(=\)

\(\ds \sum_i \frac {\condprob {\paren {A \cap B_i} } C } {\condprob {B_i} C} \, \condprob {B_i} C\)

Definition of $Q_C$

\(\ds \)

\(=\)

\(\ds \sum_i \frac {\map \Pr {A \cap B_i \cap C} } {\map \Pr C} \frac {\map \Pr C} {\map \Pr {B_i \cap C} } \, \condprob {B_i} C\)

Definition of Conditional Probability for $\condprob {A \cap B_i} C$

\(\ds \)

\(=\)

\(\ds \sum_i \frac {\map \Pr {A \cap B_i \cap C} } {\map \Pr {B_i \cap C} } \, \condprob {B_i} C\)

cancelling $\map \Pr C$ from top and bottom

\(\ds \)

\(=\)

\(\ds \sum_i \condprob A {B_i \cap C} \, \condprob {B_i} C\)

Definition of Conditional Probability for $\condprob A {B_i \cap C}$

\(\ds \)

\(=\)

\(\ds \sum_i \condprob A {B_i \cap C} \, \map \Pr {B_i}\)

$C$ and $B_i$ are independent

\(\ds \)

\(=\)

\(\ds \sum_i \condprob A {C \cap B_i} \, \map \Pr {B_i}\)

Intersection is Commutative

$\blacksquare$