Spacing Limit Theorem

Theorem
Let $X_{(i)}$ be the $i^{th}$ ordered statistic of $N$ samples from a continuous random distribution with density function $f_X(x)$

Then the spacing between the ordered statistics given $X_{(i)}$ converges in distribution to exponential for sufficiently large sampling according to:
 * $\displaystyle N\left(X_{(i+1)}-X_{(i)}\right)\xrightarrow{D} Exp\left(\frac{1}{f\left(X_{(i)}\right)}\right)$ as $N \to \infty$ for $i=1,2,3,\ldots,N-1$

Proof
Given $i$ and $N$, the ordered statistic $X_{(i)}$ has the probability density function:
 * $\displaystyle f_{X_{(i)}}(x|i,N)=\frac{N!}{(i-1)!(N-i)!}F_X(x)^{i-1}(1-F_X(x))^{N-i}f_X(x)$

where $F_X(x)$ is the cumulative distribution function of $X$.

Let $Y_i=N(X_{(i+1)}-X_{(i)})$ be an independent spacing variable valid for $i=1,2,3,\ldots,N-1$ which is always positive.

The joint density function of both $X_{(i)}$ and $Y_{i}$ is then
 * $\displaystyle f_{X_{(i)},Y_i}(x,y|i,N)=\frac{(N-1)!}{(i-1)!(N-i-1)!}F_X(x)^{i-1}\left(1-F_X\left(x+\frac y N\right)\right)^{N-i-1}f_X(x)f_X\left(x+\frac y N\right)$

The conditional density function of $Y_{i}$ given $X_{(i)}$ is
 * $\displaystyle f_{Y_i}=\frac{f_{X_{(i)},Y_i}}{f_{X_{(i)}}}$

which turns into
 * $\displaystyle f_{Y_i}(y|x=X_{(i)},i,N)=\frac{N-i}{N}\frac{\left(1-F_X\left(x+\frac y N\right)\right)^{N-i-1}}{\left(1-F_X(x)\right)^{N-i}}f_X\left(x+\frac y N\right)$

The conditional cumulative function of $Y_{i}$ given $X_{(i)}$ is:
 * $\displaystyle F_{Y_i}(y|x=X_{(i)},i,N)=1-\left(\frac{1-F_X\left(x+\frac y N\right)}{1-F_X(x)}\right)^{N-i}$

The following Taylor expansion in $y$ is an approximation of $F_X\left(x+\frac y N\right)$:
 * $\displaystyle F_X\left(x+\frac y N\right)=F_X(x)+f_X(x)\frac y N + O\left(N^{-2}\right)$

Inserting this produces
 * $\displaystyle F_{Y_i}(y|x=X_{(i)},i,N)=1-\left(1-\frac{f_X(x)y}{N(1-F_X(x))}+ O\left(N^{-2}\right)\right)^{N-i}$

The limit as $N$ gets large is the exponential function:
 * $\displaystyle F_{Y_i}(y|x=X_{(i)},i,N)=1-e^{-f_X(x)y\frac{1-\frac i N}{1-F_X(x)}}+ O\left(N^{-1}\right)$

The distribution of $F_X$ is uniform by definition of a random pick:
 * $\displaystyle F_X(x)\sim U(0,1)$

Therefore $i$ is uniformly distributed as well:
 * $\displaystyle F_X(X_{(i)})\approx \frac i N$

The limit of $F_{Y_i}$ is then
 * $\displaystyle \lim_{N\rightarrow\infty} F_{Y_i}(y|x=X_{(i)},i,N)=1-e^{-f_X(x)y}$