Spacing Limit Theorem

Theorem
Let $X_1,X_2,X_3,\ldots,X_N$ be a sequence of independent identically distributed random variables chosen from a random distribution with density function $f_X(x)$.

Let $X_{(1)},X_{(2)},X_{(3)},\ldots,X_{(N)}$ be these variables in increasing order.

Then the spacing between the ordered statistics converges in distribution to Exponential for sufficiently large sampling according to:
 * $\displaystyle N\left(X_{(i+1)}-X_{(i)}\right)\xrightarrow{D} Exp\left(\frac{1}{f\left(X_{(i)}\right)}\right)$ as $N \to \infty$

For $i=1,2,3,\ldots,N-1$.

This result allows for easy identification of the shape of the density function from the random variables.

Proof
Given $i$ and $N$, the ordered statistic $X_{(i)}$ has the probability density function:
 * $\displaystyle f_{X_{(i)}}(x|i,N)=\frac{N!}{(i-1)!(N-i)!}F_X(x)^{i-1}(1-F_X(x))^{N-i}f_X(x)$

where $F_X(x)$ is the cumulative distribution function of $X$.

Let $Y_i=N(X_{(i+1)}-X_{(i)})$ be an independent spacing variable valid for $i=1,2,3,\ldots,N-1$ which is always positive.

The joint density function of both $X_{(i)}$ and $Y_{i}$ is then
 * $\displaystyle f_{X_{(i)},Y_i}(x,y|i,N)=\frac{(N-1)!}{(i-1)!(N-i-1)!}F_X(x)^{i-1}\left(1-F_X\left(x+\frac y N\right)\right)^{N-i-1}f_X(x)f_X\left(x+\frac y N\right)$

The conditional density function of $Y_{i}$ given $X_{(i)}$ is $f_{Y_i}=\frac{f_{X_{(i)},Y_i}}{f_{X_{(i)}}}$:
 * $\displaystyle f_{Y_i}(y|x=X_{(i)},i,N)=\frac{N-i}{N}\frac{\left(1-F_X\left(x+\frac y N\right)\right)^{N-i-1}}{\left(1-F_X(x)\right)^{N-i}}f_X\left(x+\frac y N\right)$

The conditional cumulative function of $Y_{i}$ given $X_{(i)}$ is:
 * $\displaystyle F_{Y_i}(y|x=X_{(i)},i,N)=1-\left(\frac{1-F_X\left(x+\frac y N\right)}{1-F_X(x)}\right)^{N-i}$

Using the Taylor expansion:
 * $\displaystyle F_X\left(x+\frac y N\right)=F_X(x)+f_X(x)\frac y N + O\left(N^{-2}\right)$

produces
 * $\displaystyle F_{Y_i}(y|x=X_{(i)},i,N)=1-\left(1-\frac{f_X(x)y}{N(1-F_X(x))}+ O\left(N^{-2}\right)\right)^{N-i}$

The exponent limit is:
 * $\displaystyle F_{Y_i}(y|x=X_{(i)},i,N)=1-e^{-f_X(x)y\frac{1-\frac i N}{1-F_X(x)}}+ O\left(N^{-1}\right)$

Since $F_X(x)\approx \frac i N$ the limit of $F_{Y_i}$ is then
 * $\displaystyle \lim_{N\rightarrow\infty} F_{Y_i}(y|x=X_{(i)},i,N)=1-e^{-f_X(x)y}$