Markov Chain/Examples

Examples of Markov Chains

The simplest Markov chain $\sequence {X_B}$ is the case where the state space $S$ is a Boolean domain, say $S = \set {0, 1}$.

Its transition probabilities are hence $p_{00}$, $p_{01}$, $p_{10}$ and $p_{11}$.

Hence for example:

$p_{01}$ is the probability that $\sequence {X_B}$ changes from state $0$ to state $1$

$p_{11}$ is the probability that $\sequence {X_B}$ remains in state $1$ if it is already there.

Its transition matrix is of the form:

$\begin {pmatrix} p_{00} & p_{01} \\ p_{10} & p_{11} \end {pmatrix}$

where:

$\ds p_{00} + p_{01}$

$=$

$\ds 1$

$\ds p_{10} + p_{11}$

$=$

$\ds 1$