Definition:Entropy of Finite Partition

Definition
Let $\struct {\Omega, \Sigma, \Pr}$ be a probability space.

Let $\xi$ be a finite partition of $\Omega$.

The entropy of $\xi$ is defined as:
 * $\ds \map H \xi := \sum_{A \mathop \in \xi} \map \phi {\map \Pr A}$

where $\phi : \closedint 0 1 \to \R _{\ge 0}$ is defined by:
 * $\map \phi x := \begin {cases}

0 & : x = 0 \\ -x \map \ln x & : x \in \hointl 0 1 \end {cases}$

Here $\ln$ denotes the natural logarithm.

Also see

 * Definition:Entropy of Finite Sub-Sigma-Algebra
 * Definition:Conditional Entropy of Finite Partitions