Sum of Expectations of Independent Trials

From ProofWiki
Jump to navigation Jump to search

Theorem

Let $\EE_1, \EE_2, \ldots, \EE_n$ be a sequence of experiments whose outcomes are independent of each other.

Let $X_1, X_2, \ldots, X_n$ be discrete random variables on $\EE_1, \EE_2, \ldots, \EE_n$ respectively.


Let $\expect {X_j}$ denote the expectation of $X_j$ for $j \in \set {1, 2, \ldots, n}$.


Then we have, whenever both sides are defined:

$\displaystyle \expect {\sum_{j \mathop = 1}^n X_j} = \sum_{j \mathop = 1}^n \expect {X_j}$


That is, the sum of the expectations equals the expectation of the sum.


Proof 1

The proof proceeds by induction on the number of terms $n$ in the sum.

For all $n \in \Z_{\ge 0}$, let $\map P n$ be the proposition:

$\displaystyle \expect {\sum_{j \mathop = 1}^n X_j} = \sum_{j \mathop = 1}^n \expect {X_j}$


Basis for the Induction

$\map P 1$ is the case:

$\displaystyle \expect {\sum_{j \mathop = 1}^1 X_j} = \sum_{j \mathop = 1}^1 \expect {X_j}$

That is:

$\expect {X_1} = \expect {X_1}$

which is tautologically true.


This is our basis for the induction.


Induction Hypothesis

Now it needs to be shown that if $\map P k$ is true, where $k \ge 1$, then it logically follows that $\map P {k + 1}$ is true.


So this is the induction hypothesis: $\displaystyle \expect {\sum_{j \mathop = 1}^k X_j} = \sum_{j \mathop = 1}^k \expect {X_j}$


from which it is to be shown that: $\displaystyle \expect {\sum_{j \mathop = 1}^{k + 1} X_j} = \sum_{j \mathop = 1}^{k + 1} \expect {X_j}$


Induction Step

This is our induction step:


Denote $Y = \displaystyle \sum_{j \mathop = 1}^k X_j$

Then we compute:

\(\ds \expect {\sum_{j \mathop = 1}^{k + 1} X_j}\) \(=\) \(\ds \expect {Y + X_{k + 1} }\)
\(\ds \) \(=\) \(\ds \sum_{y \mathop + x_{k + 1} \mathop \in \R} \paren {y + x_{k + 1} } \map \Pr {Y + X_{k + 1} = y + x_{k + 1} }\) Definition of Expectation
\(\ds \) \(=\) \(\ds \sum_{y \mathop \in \R} \sum_{x_{k + 1} \mathop \in \R} \paren {y + x_{k + 1} } \map \Pr {Y = y, X_{k + 1} = x_{k + 1} }\) Definition of Joint Probability Mass Function
\(\ds \) \(=\) \(\ds \sum_{y \mathop \in \R} \sum_{x_{k + 1} \mathop \in \R} \paren {y + x_{k + 1} } \map \Pr {Y = y} \, \map \Pr {X_{k + 1} = x_{k + 1} }\) Independence of $Y$ and $X_{k + 1}$
\(\ds \) \(=\) \(\ds \sum_{y \mathop \in \R} \sum_{x_{k + 1} \mathop \in \R} y \, \map \Pr {Y = y} \, \map \Pr {X_{k + 1} = x_{k + 1} } + \sum_{y \mathop \in \R} \sum_{x_{k + 1} \mathop \in \R} x_{k + 1} \, \map \Pr {Y = y} \, \map \Pr {X_{k + 1} = x_{k + 1} }\) splitting the summation
\(\ds \) \(=\) \(\ds \sum_{y \mathop \in \R} y \, \map \Pr {Y = y} + \sum_{x_{k + 1} \mathop \in \R} x_{k + 1} \, \map \Pr {X_{k + 1} = x_{k + 1} }\) because $\displaystyle \sum_{x_{k + 1} \mathop \in \R} \map \Pr {X_{k + 1} = x_{k + 1} } = 1 = \sum_{y \mathop \in \R} \map \Pr {Y = y}$
\(\ds \) \(=\) \(\ds \expect Y + \expect {X_{k + 1} }\) Definition of Expectation
\(\ds \) \(=\) \(\ds \sum_{j \mathop = 1}^{k + 1} \expect {X_j}\) by the Induction Hypothesis

The result follows by the Principle of Mathematical Induction.

$\blacksquare$


Proof 2

The proof proceeds by induction on the number of terms $n$ in the sum.

For all $n \in \Z_{\ge 0}$, let $\map P n$ be the proposition:

$\ds \expect {\sum_{j \mathop = 1}^n X_j} = \sum_{j \mathop = 1}^n \expect {X_j}$


Basis for the Induction

$\map P 1$ is the case:

$\ds \expect {\sum_{j \mathop = 1}^1 X_j} = \sum_{j \mathop = 1}^1 \expect {X_j}$

That is:

$\expect {X_1} = \expect {X_1}$

which is tautologically true.


This is our basis for the induction.


Induction Hypothesis

Now it needs to be shown that if $\map P k$ is true, where $k \ge 1$, then it logically follows that $\map P {k + 1}$ is true.


So this is the induction hypothesis: $\ds \expect {\sum_{j \mathop = 1}^k X_j} = \sum_{j \mathop = 1}^k \expect {X_j}$


from which it is to be shown that: $\ds \expect {\sum_{j \mathop = 1}^{k + 1} X_j} = \sum_{j \mathop = 1}^{k + 1} \expect {X_j}$


Induction Step

This is the induction step:


Denote the random variable $Y = \ds \sum_{j \mathop = 1}^k X_j$.

Then we compute:

\(\ds \expect {\sum_{j \mathop = 1}^{k + 1} X_j}\) \(=\) \(\ds \expect {Y + X_{k + 1} }\)
\(\ds \) \(=\) \(\ds \expect Y + \expect {X_{k + 1} }\) Linearity of Expectation Function
\(\ds \) \(=\) \(\ds \sum_{j \mathop = 1}^{k + 1} \expect {X_j}\) by the Induction Hypothesis

The result follows by the Principle of Mathematical Induction.

$\blacksquare$