Strong Separation Theorem

Theorem
If $C\subset \mathbb{R}^\ell$ is closed and convex and $D=\lbrace \mathbf{v}\rbrace \subset C^c$, then $C$ and $D$ can be strongly separated.

Lemma 1
For any $\mathbf{y}\neq 0$ and any $\mathbf{z}\in H_{\mathbf{y}}^<(r)$, $r=\mathbf{y}\mathbf{y}$, the line segment joining $\mathbf{y}$ and $\mathbf{z}$ contains points with lengths strictly less than $\mathbf{y}$.

Proof of Lemma 1
Since $\mathbf{z}\in H_\mathbf{y}(r)^<$, $\mathbf{z}=\mathbf{x}-s\mathbf{y}$ for some $\mathbf{x}\bot\mathbf{y}$ and $s>0$.

Let $f(\gamma)=\lVert \gamma\mathbf{y}+(1-\gamma)\mathbf{z}\rVert^2$ for $\gamma\in [0,1]$.

We show that $f(\gamma)<\lVert \mathbf{y} \rVert^2$ as $\gamma\rightarrow 1$.

Expand $f(\gamma)$:

Since $f'(1)=2(1+s)>0$, $f(\gamma)0$ such that $K=\mbox{cl}(B_n(\mathbf{v}))\cap C\neq \emptyset$.

Since $C$ is closed, $K$ is closed.

Since $B_n(\mathbf{v})$ is bounded, $K$ is bounded.

Since $K$ is closed and bounded, $K$ is compact.

Let $f(\mathbf{x})=\lVert \mathbf{x} \rVert$.

Since $\lVert \mathbf{x} \rVert^2$ and $\sqrt{x}:\mathbb{R}_+\rightarrow\mathbb{R}_+$ are both continuous, $f(\mathbf{x})$ is continuous.

Since $K$ is compact and $f(\mathbf{x})$ is continuous, $f(\mathbf{x})$ achieves its minimum at $\mathbf{y}\in C$ by the Weistrass Theorem.

Let $L(\mathbf{x})=\mathbf{y}\mathbf{x}$ and $r=\mathbb{y}\mathbb{y}$.

We show that $C\subset L^{-1}([r,\infty])$.

Suppose $C$ is not a subset of $L^{-1}([r,\infty))$.

There exists $\mathbf{z}\in C\cap L^{-1}((-\infty, r))$, so $\mathbf{z}\in H_\mathbf{y}(r)$.

As $\gamma\rightarrow 1$, there exists a $0<\gamma<1$ such that $\lVert \mathbf{y}\gamma\mathbf{z}\rVert<\lVert \mathbf{y}\rVert$ by Lemma 1.

Since $C$ is convex, $\mathbf{y}\in C$ and $\mathbf{z}\in C$, convex combination $\mathbf{y}\gamma \mathbf{z}\in C$.

Since $\mathbf{y}$ is a minimmizer of $\lVert \mathbf{x}\rVert$, $\lVert \mathbf{y}\rVert\leq \lVert \mathbf{y}\gamma\mathbf{z}\rVert$, a contradiction.

For strong separation, let $\epsilon0$, $\mathbf{v}=\mathbf{0}\subset L^{-1}((-\infty,r/2-\epsilon])$.