Conditional Entropy Decreases if More Given

Theorem
Let $\struct {\Omega, \Sigma, \Pr}$ be a probability space.

Let $\AA, \CC, \DD \subseteq \Sigma$ be finite sub-$\sigma$-algebras.

Then:
 * $\CC \subseteq \DD \implies \map H {\AA \mid \CC} \ge \map H {\AA \mid \DD}$

where:
 * $\map H {\cdot \mid \cdot}$ denotes the conditional entropy

Proof
Let:
 * $\xi := \map \sigma \AA$
 * $\eta := \map \sigma \CC$
 * $\gamma := \map \sigma \DD$

By Generating Partition Preserves Order, $\CC \subseteq \DD$ implies:
 * $\eta \le \gamma$

where $\le$ denotes the refinement.

Observe that for all $B \in \eta$ and $D \in \gamma$:
 * $\map \mu {B \cap D} = \begin{cases}

\map \mu D &: D \subseteq B \\ 0 &: B \cap D = \O \end{cases}$

Therefore, for all $A\in\xi$ and $B\in\eta$ such that $\map\mu B >0$:

Since $\phi$ is concave, by Jensen's inequality: