Weierstrass Approximation Theorem

Theorem

Let $f$ be a real function which is continuous on the closed interval $\Bbb I = \closedint a b$.

Then $f$ can be uniformly approximated on $\Bbb I$ by a polynomial function to any given degree of accuracy.

Complex Case

Let $f : \Bbb I \to \C$ be a continuous complex function, where $\Bbb I = \closedint a b$ is a closed real interval.

Let $\epsilon \in \R_{>0}$.

Then there exists a complex polynomial function $p : \Bbb I \to \C$ such that:

$\norm { p - f }_\infty < \epsilon$

where $\norm { f }_\infty$ denotes the supremum norm of $f$ on $\Bbb I$.

Proof 1

Let $\map f t: \Bbb I = \closedint a b \to \R$ be a continuous function.

Introduce $\map x t$ with a rescaled domain:

$\map f t \mapsto \map x {a + t \paren {b - a} } : \closedint a b \to \closedint 0 1$

From now on we will work with $x: \closedint 0 1 \to \R$, which is also continuous.

Let $n \in \N$.

For $t \in \closedint 0 1$ consider the Bernstein polynomial:

$\ds \map {B_n x} t = \sum_{k \mathop = 0}^n \map x {\frac k n} \binom n k t^k \paren {1 - t}^{n - k}$

For $t \in \closedint 0 1$, $0 \le k \le n$, let:

$\map {p_{n, k} } t := \dbinom n k t^k \paren {1 - t}^{n - k}$

By the binomial theorem:

$\ds \sum_{k \mathop = 0}^n \map {p_{n, k} } t = 1$

Lemma 1

$\ds \sum_{k \mathop = 0}^n k \map {p_{n, k} } t = n t$

$\Box$

Lemma 2

$\ds \sum_{k \mathop = 0}^n \paren {k - n t}^2 \map {p_{n, k} } t = n t \paren {1 - t}$

$\Box$

Now we construct the estimates.

 $\ds \sum_{k \mathop : \size {\frac k n \mathop - t} \ge \delta}^n \map {p_{n, k} } t$ $\le$ $\ds \sum_{k \mathop : \size {\frac k n \mathop - t} \ge \delta}^n \map {p_{n, k} } t \frac {\paren {k - n t}^2} {\delta^2 n^2}$ as $\dfrac {\paren {k - n t}^2} {\delta^2 n^2} \ge 1$ $\ds$ $\le$ $\ds \frac 1 {n^2 \delta^2} \sum_{k \mathop = 0}^n \paren{k - n t}^2 \map {p_{n, k} } t$ $\ds$ $=$ $\ds \frac {t \paren {1 - t} } {n \delta^2}$ Lemma 2 $\ds$ $\le$ $\ds \frac 1 {4 n \delta^2}$ $\forall t \in \closedint 0 1: \dfrac 1 4 \ge t \paren {1 - t}$

Here $\ds \sum_{k \mathop : \size {\frac k n \mathop - t} \ge \delta}^n$ denotes the summation over those values of $k \in \N$, $k \le n$, which satisfy the inequality $\size {\dfrac k n - t} \ge \delta$.

For some $\delta > 0$ denote:

$\ds \map {\omega_\delta} x := \sup_{\size {t - s} < \delta} \size {\map x s - \map x t}$

Then:

 $\ds \size {\map {B_n x} t - \map x t}$ $=$ $\ds \size {\map {B_n x} t - \map x t \sum_{k \mathop = 0}^n \map {p_{n, k} } t}$ $\ds$ $=$ $\ds \size {\sum_{k \mathop = 0}^n \map x {\frac k n} \map {p_{n, k} } t - \map x t \sum_{k \mathop = 0}^n \map {p_{n, k} } t}$ $\ds$ $\le$ $\ds \sum_{k \mathop = 0}^n \size {\map x {\frac k n} - \map x t} \map {p_{n, k} } t$ as $\map {p_{n, k} } t \ge 0$ $\ds$ $=$ $\ds \sum_{k \mathop : \size {\frac k n \mathop - t} < \delta}^n \size {\map x {\frac k n} - \map x t} \map {p_{n, k} } t + \sum_{k \mathop : \size {\frac k n \mathop - t} \ge \delta}^n \size {\map x {\frac k n} - \map x t} \map {p_{n, k} } t$ $\ds$ $\le$ $\ds \map {\omega_\delta} x \sum_{k \mathop : \size {\frac k n \mathop - t} < \delta}^n \map {p_{n, k} } t + 2 \norm x_\infty \frac 1 {4 n \delta^2}$ $\ds$ $\le$ $\ds \map {\omega_\delta} x \cdot 1 + \frac {\norm x_\infty} {2 n \delta^2}$

where $\norm {\,\cdot \,}_\infty$ denotes the supremum norm.

Let $\epsilon > 0$.

We choose $\delta > 0$ such that $\map {\omega_\delta} x < \dfrac \epsilon 2$.

Choose $n > \dfrac {\norm x_\infty} {\epsilon \delta^2}$

Then:

$\norm {\map {B_n x} t - \map x t}_\infty < \epsilon$.

$\blacksquare$

Proof 2

Without loss of generality, assume $\Bbb I = \closedint 0 1$

For each $n \in \N$, let:

$\ds \map {P_n} x := \sum_{k \mathop = 0}^n \map f {\dfrac k n } \dbinom n k x^k \paren {1 - x}^{n - k}$

We shall show that $\lim_{n \to \infty} \norm { P_n - f}_\infty = 0$.

Let $\epsilon \in \R_{>0}$.

By Heine-Cantor Theorem, there is a $\delta \in \R_{>0}$ such that:

$\forall x,y \in \Bbb I : \size {x - y} \le \delta \implies \size {\map f x - \map f y} \le \epsilon$

Let $p \in \Bbb I$.

Let $Z_n$ be a random variable such that:

$\ds n Z_n \sim \Binomial n p$

where $\Binomial n p$ denotes the binomial distribution with parameters $n$ and $p$.

Observe that:

 $\ds \expect {\map f {Z_n} }$ $=$ $\ds \sum_{k \mathop = 0}^n \map f {\dfrac k n} \map \Pr {Z_n = \dfrac k n}$ Definition of Expectation of Discrete Random Variable $\ds$ $=$ $\ds \sum_{k \mathop = 0}^n \map f {\dfrac k n} \map \Pr {n Z_n = k}$ $\ds$ $=$ $\ds \sum_{k \mathop = 0}^n \map f {\dfrac k n} \dbinom n k p^k \paren {1 - p}^{n - k}$ Definition of Binomial Distribution $\text {(1)}: \quad$ $\ds$ $=$ $\ds \map {P_n} p$

Furthermore:

 $\ds \map \Pr {\size { Z_n - p} > \delta }$ $\le$ $\ds \dfrac {\expect {\size {Z_n - p } ^2} } {\delta^2}$ Chebyshev's Inequality $\ds$ $=$ $\ds \dfrac {\expect {\size {n Z_n - n p} ^2} } {\delta^2 n^2}$ Expectation is Linear $\ds$ $=$ $\ds \dfrac {\expect {\size {n Z_n - \expect {n Z_n} } ^2} } {\delta^2 n^2}$ Expectation of Binomial Distribution $\ds$ $=$ $\ds \dfrac {\var {n Z_n} } {\delta^2 n^2}$ Definition of Variance $\ds$ $=$ $\ds \dfrac {p \paren {1 - p} } {\delta^2 n}$ Variance of Binomial Distribution $\ds$ $\le$ $\ds \dfrac 1 {4 \delta^2 n}$ Cauchy's Mean Theorem

On the other hand:

 $\ds \size {\map f {Z_n} - \map f p}$ $\le$ $\ds \epsilon + \size {\map f {Z_n} - \map f p } \chi_{\set {\size {Z_n - p} > \delta } }$ $\ds$ $\le$ $\ds \epsilon + 2 \norm f_\infty \chi_{\set {\size { Z_n - p } > \delta} }$

Therefore:

 $\ds \size {\map {P_n} p - \map f p}$ $=$ $\ds \size {\expect {\map f { Z_n } } - \map f p}$ by $(1)$ $\ds$ $=$ $\ds \size {\expect {\map f {Z_n} } - \expect {\map f p} }$ Expectation of Almost Surely Constant Random Variable $\ds$ $=$ $\ds \size {\expect {\map f {Z_n} - \map f p} }$ Expectation is Linear $\ds$ $\le$ $\ds \expect {\size {\map f {Z_n} - \map f p} }$ $\ds$ $\le$ $\ds \epsilon + 2 \norm f_\infty \map \Pr {\size {Z_n - p} > \delta}$ $\ds$ $\le$ $\ds \epsilon + \dfrac {\norm f_\infty} {2 \delta^2 n}$

Thus for all $n \in \N_{> 2 \delta^2 / \norm f_\infty}$ we have:

$\size {\map {P_n} p - \map f p} \le 2 \epsilon$

As the above is true for all $p \in \Bbb I$, we have:

$\forall n \in \N_{> 2 \delta^2 / \norm f_\infty} : \norm { P_n - f}_\infty \le 2 \epsilon$

$\blacksquare$

Proof 3

Let $\AA \subseteq \map C {\Bbb I, \R}$ be the set of real polynomial functions.

$\AA$ is a subalgebra of $\map C {\Bbb I, \R}$ because polynomials over $\R$ form an algebra over $\R$.

Let $I$ denote the identity mapping on $\Bbb I$, i.e.:

$\forall x \in \Bbb I : \map I x = x$

Then $I \in \AA$.

Thus $\AA$ separates the points of $\Bbb I$, since trivially:

$\forall x,y \in \Bbb I : x \ne y \implies \map I x \ne \map I y$

It is also clear that $1 \in \AA$.

Therefore the claim follows from Stone-Weierstrass Theorem.

$\blacksquare$

Also known as

This result is also seen referred to as Weierstrass's theorem, but as there are a number of results bearing Karl Weierstrass's name, it makes sense to be more specific.

Source of Name

This entry was named for Karl Weierstrass.

Historical Note

The Weierstrass Approximation Theorem has been demonstrated to have far-reaching and important effects in every aspect of the field of analysis.

It has also been given a significant generalisation by Marshall Harvey Stone.