Strong Separation Theorem

Theorem
Let $C \subset \R^\ell$ be closed and convex.

Let $D = \left\{{\mathbf v}\right\} \subset C^c$.

Then $C$ and $D$ can be strongly separated.

Lemma 1
For any $\mathbf y \ne 0$ and any $\mathbf z \in H_{\mathbf y}^< \left({r}\right)$, $r = \mathbf y \mathbf y$, the line segment joining $\mathbf y$ and $\mathbf z$ contains points with lengths strictly less than $\mathbf y$.

Proof of Lemma 1
Since $\mathbf z \in H_\mathbf y \left({r}\right)^<$:
 * $\mathbf z = \mathbf x - s \mathbf y$

for some $\mathbf x \bot \mathbf y$ and $s > 0$.

Let:
 * $f \left({\gamma}\right) = \left\lVert{\gamma \mathbf y + \left({1 - \gamma}\right) \mathbf z}\right\rVert^2$

for $\gamma \in \left[{0 \,.\,.\, 1}\right]$.

We show that:
 * $f \left({\gamma}\right) < \left\lVert{\mathbf y}\right\rVert^2$

as $\gamma \to 1$.

Expand $f \left({\gamma}\right)$:

Since $f' \left({1}\right) = 2 \left({1 + s}\right) > 0$:
 * $f \left({\gamma}\right) < f \left({1}\right)$

as $\gamma \to 1$.

Since $f \left({1}\right) = \mathbf y \mathbf y = \left\lVert{\mathbf y}\right\rVert^2$:
 * $f \left({\gamma}\right) < \left\lVert{\mathbf y}\right\rVert^2$

as $\gamma \to 1$.

Proof of Theorem
Translating $C$ and $D = \left\{ {\mathbf v}\right\}$ by $-\mathbf v$:

We show that the theorem holds for $\mathbf v = 0$.

Pick $n > 0$ such that:
 * $K = \operatorname{cl} \left({B_n \left({\mathbf v}\right)}\right) \cap C \ne \varnothing$

Since $C$ is closed, $K$ is closed.

Since $B_n \left({\mathbf v}\right)$ is bounded, $K$ is bounded.

Since $K$ is closed and bounded, $K$ is compact.

Let $f \left({\mathbf x}\right) = \left\lVert{\mathbf x}\right\rVert$.

Since $\left\lVert{\mathbf x}\right\rVert^2$ and $\sqrt x: \R_+ \to \R_+$ are both continuous, $f \left({\mathbf x}\right)$ is continuous.

Since $K$ is compact and $f \left({\mathbf x}\right)$ is continuous, $f \left({\mathbf x}\right)$ achieves its minimum at $\mathbf y \in C$ by the Weierstrass Theorem.

Let $L \left({\mathbf x}\right) = \mathbf y \mathbf x$ and $r = \mathbf y \mathbf y$.

We show that:
 * $C \subset L^{-1} \left({\left[{r \,.\,.\, \to}\right)}\right)$

Suppose $C$ is not a subset of $L^{-1} \left({\left[{r \,.\,.\, \to}\right)}\right)$.

There exists $\mathbf z \in C \cap L^{-1} \left({\left({-\infty \,.\,.\, r}\right)}\right)$

So:
 * $\mathbf z \in H_\mathbf y \left({r}\right)$

As $\gamma \to 1$, there exists a $0 < \gamma < 1$ such that:
 * $\left\lVert{\mathbf y \, \gamma \, \mathbf z}\right\rVert < \left\lVert{\mathbf y}\right\rVert$

by Lemma 1.

Since $C$ is convex, $\mathbf y \in C$ and $\mathbf z \in C$, convex combination $\mathbf y \, \gamma \, \mathbf z \in C$.

Since $\mathbf y $ is a minimmizer of $\left\lVert{\mathbf x}\right\rVert$:
 * $\left\lVert{\mathbf y}\right\rVert \le \left\lVert{\mathbf y \, \gamma \, \mathbf z}\right\rVert$

which is a contradiction.

For strong separation, let $\epsilon < r / 2$.

We have:
 * $C \subset L^{-1} \left({\left[{r \,.\,.\, \to}\right)}\right) \subset L^{-1} \left({ \left[{r/2 + \epsilon \,.\,.\, \infty}\right)}\right)$

Since:
 * $L \left({\mathbf 0}\right) = \mathbf y \mathbf 0 = 0$

and:
 * $r/2 - \epsilon > 0$

it follows that:
 * $\mathbf v = \mathbf 0 \subset L^{-1} \left({\left({-\infty \,.\,.\, r/2 - \epsilon}\right]}\right)$