Chapman-Kolmogorov Equation

Theorem
Let $X$ be a discrete state-space Markov Chain with with $n$-step transition probability matrix:
 * $\mathbf P^{\left({n} \right)} = \left[{p^{\left({n}\right)} \left({j, k}\right)}\right]_{j, k \mathop \in S}$

where:
 * $p^{\left({n}\right)} \left({j, k}\right) = \mathbb P \left[{X_{m + n} = k \mid X_m = j}\right] = {p_{j k} }^{\left({n}\right)}$ is the $n$-step transition probability.

Then:
 * $\mathbf P^{\left({n + m}\right)} = \mathbf P^{\left({n}\right)} \mathbf P^{\left({m}\right)}$

or equivalently:
 * $\displaystyle {p_{i j} }^{\left({n + m}\right)} = \sum_{k \mathop \in S} {p_{i k} }^{\left({n}\right)} {p_{k j} }^{\left({m}\right)}$

Proof
We consider the conditional probability on the left hand side: