Definition:Entropy of Finite Partition

Definition
Let $\AA \subseteq \Sigma$ be a sub-$\sigma$-algebra. Let $\struct {\Omega, \Sigma, \Pr}$ be a probability space.

Let $\xi$ be a finite partition of $\Omega$.

The entropy of $\xi$ is defined as:
 * $\ds \map H \xi := -\sum_{A \mathop \in \xi} \map \Pr A \map \ln {\map \Pr A}$

where $\ln$ denotes the natural logarithm.