Conditional Entropy of Join as Sum

Theorem
Let $\struct {\Omega, \Sigma, \Pr}$ be a probability space.

Let $\AA, \CC, \DD \subseteq \Sigma$ be finite sub-$\sigma$-algebras.

Then:
 * $\ds \map H {\AA \vee \CC \mid \DD} = \map H {\AA \mid \DD} + \map H {\CC \mid \AA \vee \DD} $

where:
 * $\map H {\cdot \mid \cdot}$ denotes the conditional entropy
 * $\vee$ denotes the join

Proof
Consider the generated finite partitions:
 * $\xi := \map \xi \AA$
 * $\eta := \map \xi \CC$
 * $\gamma := \map \xi \DD$

By, we shall show:
 * $\map H {\xi \vee \eta \mid \gamma} = \map H {\xi \mid \gamma} + \map H {\eta \mid \xi \vee \gamma}$

Then:

Now:

and: