Weierstrass Approximation Theorem
Theorem
Let $f$ be a real function which is continuous on the closed interval $\Bbb I = \closedint a b$.
Then $f$ can be uniformly approximated on $\Bbb I$ by a polynomial function to any given degree of accuracy.
Complex Case
Let $f : \Bbb I \to \C$ be a continuous complex function, where $\Bbb I = \closedint a b$ is a closed real interval.
Let $\epsilon \in \R_{>0}$.
Then there exists a complex polynomial function $p : \Bbb I \to \C$ such that:
- $\norm { p - f }_\infty < \epsilon$
where $\norm { f }_\infty$ denotes the supremum norm of $f$ on $\Bbb I$.
Proof 1
Let $\map f t: \Bbb I = \closedint a b \to \R$ be a continuous function.
Introduce $\map x t$ with a rescaled domain:
- $\map f t \mapsto \map x {a + t \paren {b - a} } : \closedint a b \to \closedint 0 1$
From now on we will work with $x: \closedint 0 1 \to \R$, which is also continuous.
Let $n \in \N$.
For $t \in \closedint 0 1$ consider the Bernstein polynomial:
- $\ds \map {B_n x} t = \sum_{k \mathop = 0}^n \map x {\frac k n} \binom n k t^k \paren {1 - t}^{n - k}$
For $t \in \closedint 0 1$, $0 \le k \le n$, let:
- $\map {p_{n, k} } t := \dbinom n k t^k \paren {1 - t}^{n - k}$
By the binomial theorem:
- $\ds \sum_{k \mathop = 0}^n \map {p_{n, k} } t = 1$
Lemma 1
- $\ds \sum_{k \mathop = 0}^n k \map {p_{n, k} } t = n t$
$\Box$
Lemma 2
- $\ds \sum_{k \mathop = 0}^n \paren {k - n t}^2 \map {p_{n, k} } t = n t \paren {1 - t}$
$\Box$
Now we construct the estimates.
\(\ds \sum_{k \mathop : \size {\frac k n \mathop - t} \ge \delta}^n \map {p_{n, k} } t\) | \(\le\) | \(\ds \sum_{k \mathop : \size {\frac k n \mathop - t} \ge \delta}^n \map {p_{n, k} } t \frac {\paren {k - n t}^2} {\delta^2 n^2}\) | as $\dfrac {\paren {k - n t}^2} {\delta^2 n^2} \ge 1$ | |||||||||||
\(\ds \) | \(\le\) | \(\ds \frac 1 {n^2 \delta^2} \sum_{k \mathop = 0}^n \paren{k - n t}^2 \map {p_{n, k} } t\) | ||||||||||||
\(\ds \) | \(=\) | \(\ds \frac {t \paren {1 - t} } {n \delta^2}\) | Lemma 2 | |||||||||||
\(\ds \) | \(\le\) | \(\ds \frac 1 {4 n \delta^2}\) | $\forall t \in \closedint 0 1: \dfrac 1 4 \ge t \paren {1 - t}$ |
Here $\ds \sum_{k \mathop : \size {\frac k n \mathop - t} \ge \delta}^n$ denotes the summation over those values of $k \in \N$, $k \le n$, which satisfy the inequality $\size {\dfrac k n - t} \ge \delta$.
For some $\delta > 0$ denote:
- $\ds \map {\omega_\delta} x := \sup_{\size {t - s} < \delta} \size {\map x s - \map x t}$
Then:
\(\ds \size {\map {B_n x} t - \map x t}\) | \(=\) | \(\ds \size {\map {B_n x} t - \map x t \sum_{k \mathop = 0}^n \map {p_{n, k} } t}\) | ||||||||||||
\(\ds \) | \(=\) | \(\ds \size {\sum_{k \mathop = 0}^n \map x {\frac k n} \map {p_{n, k} } t - \map x t \sum_{k \mathop = 0}^n \map {p_{n, k} } t}\) | ||||||||||||
\(\ds \) | \(\le\) | \(\ds \sum_{k \mathop = 0}^n \size {\map x {\frac k n} - \map x t} \map {p_{n, k} } t\) | as $\map {p_{n, k} } t \ge 0$ | |||||||||||
\(\ds \) | \(=\) | \(\ds \sum_{k \mathop : \size {\frac k n \mathop - t} < \delta}^n \size {\map x {\frac k n} - \map x t} \map {p_{n, k} } t + \sum_{k \mathop : \size {\frac k n \mathop - t} \ge \delta}^n \size {\map x {\frac k n} - \map x t} \map {p_{n, k} } t\) | ||||||||||||
\(\ds \) | \(\le\) | \(\ds \map {\omega_\delta} x \sum_{k \mathop : \size {\frac k n \mathop - t} < \delta}^n \map {p_{n, k} } t + 2 \norm x_\infty \frac 1 {4 n \delta^2}\) | ||||||||||||
\(\ds \) | \(\le\) | \(\ds \map {\omega_\delta} x \cdot 1 + \frac {\norm x_\infty} {2 n \delta^2}\) |
where $\norm {\,\cdot \,}_\infty$ denotes the supremum norm.
Let $\epsilon > 0$.
By Continuous Function on Closed Real Interval is Uniformly Continuous, $\map x t$ is uniformly continuous.
We choose $\delta > 0$ such that $\map {\omega_\delta} x < \dfrac \epsilon 2$.
Choose $n > \dfrac {\norm x_\infty} {\epsilon \delta^2}$
Then:
- $\norm {\map {B_n x} t - \map x t}_\infty < \epsilon$.
$\blacksquare$
Proof 2
Without loss of generality, assume $\Bbb I = \closedint 0 1$
For each $n \in \N$, let:
- $\ds \map {P_n} x := \sum_{k \mathop = 0}^n \map f {\dfrac k n } \dbinom n k x^k \paren {1 - x}^{n - k}$
We shall show that $\lim_{n \to \infty} \norm { P_n - f}_\infty = 0$.
Let $\epsilon \in \R_{>0}$.
By Heine-Cantor Theorem, there is a $\delta \in \R_{>0}$ such that:
- $\forall x,y \in \Bbb I : \size {x - y} \le \delta \implies \size {\map f x - \map f y} \le \epsilon $
Let $p \in \Bbb I$.
Let $Z_n$ be a random variable such that:
- $\ds n Z_n \sim \Binomial n p$
where $\Binomial n p$ denotes the binomial distribution with parameters $n$ and $p$.
Observe that:
\(\ds \expect {\map f {Z_n} }\) | \(=\) | \(\ds \sum_{k \mathop = 0}^n \map f {\dfrac k n} \map \Pr {Z_n = \dfrac k n}\) | Definition of Expectation of Discrete Random Variable | |||||||||||
\(\ds \) | \(=\) | \(\ds \sum_{k \mathop = 0}^n \map f {\dfrac k n} \map \Pr {n Z_n = k}\) | ||||||||||||
\(\ds \) | \(=\) | \(\ds \sum_{k \mathop = 0}^n \map f {\dfrac k n} \dbinom n k p^k \paren {1 - p}^{n - k}\) | Definition of Binomial Distribution | |||||||||||
\(\text {(1)}: \quad\) | \(\ds \) | \(=\) | \(\ds \map {P_n} p\) |
Furthermore:
\(\ds \map \Pr {\size { Z_n - p} > \delta }\) | \(\le\) | \(\ds \dfrac {\expect {\size {Z_n - p } ^2} } {\delta^2}\) | Chebyshev's Inequality | |||||||||||
\(\ds \) | \(=\) | \(\ds \dfrac {\expect {\size {n Z_n - n p} ^2} } {\delta^2 n^2}\) | Expectation is Linear | |||||||||||
\(\ds \) | \(=\) | \(\ds \dfrac {\expect {\size {n Z_n - \expect {n Z_n} } ^2} } {\delta^2 n^2}\) | Expectation of Binomial Distribution | |||||||||||
\(\ds \) | \(=\) | \(\ds \dfrac {\var {n Z_n} } {\delta^2 n^2}\) | Definition of Variance | |||||||||||
\(\ds \) | \(=\) | \(\ds \dfrac {p \paren {1 - p} } {\delta^2 n}\) | Variance of Binomial Distribution | |||||||||||
\(\ds \) | \(\le\) | \(\ds \dfrac 1 {4 \delta^2 n}\) | Cauchy's Mean Theorem |
On the other hand:
\(\ds \size {\map f {Z_n} - \map f p}\) | \(\le\) | \(\ds \epsilon + \size {\map f {Z_n} - \map f p } \chi_{\set {\size {Z_n - p} > \delta } }\) | ||||||||||||
\(\ds \) | \(\le\) | \(\ds \epsilon + 2 \norm f_\infty \chi_{\set {\size { Z_n - p } > \delta} }\) |
Therefore:
\(\ds \size {\map {P_n} p - \map f p}\) | \(=\) | \(\ds \size {\expect {\map f { Z_n } } - \map f p}\) | by $(1)$ | |||||||||||
\(\ds \) | \(=\) | \(\ds \size {\expect {\map f {Z_n} } - \expect {\map f p} }\) | Expectation of Almost Surely Constant Random Variable | |||||||||||
\(\ds \) | \(=\) | \(\ds \size {\expect {\map f {Z_n} - \map f p} }\) | Expectation is Linear | |||||||||||
\(\ds \) | \(\le\) | \(\ds \expect {\size {\map f {Z_n} - \map f p} }\) | ||||||||||||
\(\ds \) | \(\le\) | \(\ds \epsilon + 2 \norm f_\infty \map \Pr {\size {Z_n - p} > \delta}\) | ||||||||||||
\(\ds \) | \(\le\) | \(\ds \epsilon + \dfrac {\norm f_\infty} {2 \delta^2 n}\) |
Thus for all $n \in \N_{> 2 \delta^2 / \norm f_\infty}$ we have:
- $\size {\map {P_n} p - \map f p} \le 2 \epsilon$
As the above is true for all $p \in \Bbb I$, we have:
- $\forall n \in \N_{> 2 \delta^2 / \norm f_\infty} : \norm { P_n - f}_\infty \le 2 \epsilon$
$\blacksquare$
Proof 3
Let $\AA \subseteq \map C {\Bbb I, \R}$ be the set of real polynomial functions.
$\AA$ is a subalgebra of $\map C {\Bbb I, \R}$ because polynomials over $\R$ form an algebra over $\R$.
Let $I$ denote the identity mapping on $\Bbb I$, i.e.:
- $\forall x \in \Bbb I : \map I x = x$
Then $I \in \AA$.
Thus $\AA$ separates the points of $\Bbb I$, since trivially:
- $\forall x,y \in \Bbb I : x \ne y \implies \map I x \ne \map I y$
It is also clear that $1 \in \AA$.
Therefore the claim follows from Stone-Weierstrass Theorem.
$\blacksquare$
Also known as
This result is also seen referred to as Weierstrass's theorem, but as there are a number of results bearing Karl Weierstrass's name, it makes sense to be more specific.
Source of Name
This entry was named for Karl Weierstrass.
Historical Note
The Weierstrass Approximation Theorem has been demonstrated to have far-reaching and important effects in every aspect of the field of analysis.
It has also been given a significant generalisation by Marshall Harvey Stone.
Sources
- 1992: George F. Simmons: Calculus Gems ... (previous) ... (next): Chapter $\text {A}.33$: Weierstrass ($\text {1815}$ – $\text {1897}$)
- 1989: Ephraim J. Borowski and Jonathan M. Borwein: Dictionary of Mathematics ... (previous) ... (next): Weierstrass approximation theorem
- 1998: David Nelson: The Penguin Dictionary of Mathematics (2nd ed.) ... (previous) ... (next): Weierstrass's theorem
- 2008: David Nelson: The Penguin Dictionary of Mathematics (4th ed.) ... (previous) ... (next): Weierstrass's theorem
![]() | This page may be the result of a refactoring operation. As such, the following source works, along with any process flow, will need to be reviewed. When this has been completed, the citation of that source work (if it is appropriate that it stay on this page) is to be placed above this message, into the usual chronological ordering. If you have access to any of these works, then you are invited to review this list, and make any necessary corrections. To discuss this page in more detail, feel free to use the talk page. When this work has been completed, you may remove this instance of {{SourceReview}} from the code. |
- 2017: Amol Sasane: A Friendly Approach to Functional Analysis ... (previous) ... (next): $\S 1.3$: Normed and Banach spaces. Topology of normed spaces