On Min-Max affine approximants of convex or concave real valued functions from $\mathbb R^k$, Chebyshev equioscillation and graphics

TL;DR

This paper investigates Min-Max affine approximants of convex or concave functions on convex sets, extending Chebyshev equioscillation results and connecting to computer graphics rendering techniques.

Contribution

It proves the uniqueness of best affine approximants for convex functions on simplices and extends Chebyshev's theorem to higher dimensions.

Findings

Unique best affine approximant exists for convex functions on simplices.

Extension of Chebyshev equioscillation theorem to higher dimensions.

Connections to efficient rendering of projective transformations in graphics.

Abstract

We study Min-Max affine approximants of a continuous convex or concave function $f:\Delta\subset \mathbb R^k\xrightarrow{} \mathbb R$ where $\Delta$ is a convex compact subset of $\mathbb R^k$. In the case when $\Delta$ is a simplex we prove that there is a vertical translate of the supporting hyperplane in $\mathbb R^{k+1}$ of the graph of $f$ at the vertices which is the unique best affine approximant to $f$ on $\Delta$. For $k=1$, this result provides an extension of the Chebyshev equioscillation theorem for linear approximants. Our result has interesting connections to the computer graphics problem of rapid rendering of projective transformations.

Equations

$${\Vert g(\mathbf{x})\Vert }_{\infty (\Delta )}:=\underset{\mathbf{x}\in \Delta }{\sup }\left|g(\mathbf{x})\right|.$$

$$\underset{\left\{\mathbf{\alpha },\beta \right\}}{\min }{\Vert f(\mathbf{x})-({\mathbf{\alpha }}^T\mathbf{x}+\beta )\Vert }_{\infty (\Delta )}.$$

$$\left[\begin{array}{cc}{\mathbf{a}}_1^T& 1\\ .& 1\\ .& 1\\ .& 1\\ {\mathbf{a}}_{k+1}^T& 1\end{array}\right]\left[\begin{array}{c}{\mathbf{\alpha }}^{\prime }\\ {\beta }^{\prime }\end{array}\right]=\left[\begin{array}{c}f({\mathbf{a}}_1)\\ .\\ .\\ .\\ f({\mathbf{a}}_{k+1})\end{array}\right].$$

$${\beta }^{\prime \prime }=\{\begin{array}{ll}{\min }_{\mathbf{x}\in \Delta }(f(\mathbf{x})-({\mathbf{\alpha }}^{\prime T}\mathbf{x}+{\beta }^{\prime })),& \text{if f is convex}.\\ {\max }_{\mathbf{x}\in \Delta }(f(\mathbf{x})-({\mathbf{\alpha }}^{\prime T}\mathbf{x}+{\beta }^{\prime })),& \text{if f is concave}.\end{array}$$

$$u(t)=\sum _{j=0}^l{c}_j{u}_j(t),t\in [p,q]$$

$$U_l:=\left\{u(t):u(t)=\sum _{j=0}^l{c}_j{u}_j(t)\right\}$$

$${\Vert f-\hat{u}\Vert }_{\infty [p,q]}={\inf }_{u\in U_l}{\Vert f-u\Vert }_{\infty [p,q]}$$

$$f(x_i)-\hat{u}(x_i)=w(-1{)}^i\Vert f-\hat{u}{\Vert }_{\infty [p,q]},w=\pm 1.$$

$${\hat{h}}_l=\underset{h_l\in {\Pi }_l}{\mathrm{argmin}}{\Vert f-h_l\Vert }_{\infty [p,q]}$$

$$f(x_i)-\widehat{h_l}(x_i)=w(-1{)}^i\Vert f-\widehat{h_l}{\Vert }_{\infty [p,q]},w=\pm 1.$$

$$L(x)=f(y)+d+m(x-y),x\in [p,q].$$

$$d=f(p)-L(p)=f(p)-f(y)-d-m(p-y).$$

$$d=\frac{f(p)-f(y)}{2}-\frac{m}{2}(p-y).$$

$$d=f(q)-L(q)=f(q)-f(y)-d-m(q-y).$$

$$\frac{f(p)-f(y)}{2}-\frac{m}{2}(p-y)=\frac{f(q)-f(y)}{2}-\frac{m}{2}(q-y).$$

$$\frac{ax+b}{cx+d}==\frac{\left(\left(\frac{a}{c}\right)\left(\mathbf{x}+\frac{d}{c}-\frac{d}{c}\right)+\frac{b}{c}\right)}{\left(\mathbf{x}+\frac{d}{c}\right)}$$

$$=\frac{a}{c}+\frac{\left(\frac{b}{c}-\left(\frac{a}{c}\right)\left(\frac{d}{c}\right)\right)}{\left(\mathbf{x}+\frac{d}{c}\right)}$$

$$P:\left[\begin{array}{c}{x}_1\\ y_1\end{array}\right]\mapsto \frac{1}{d{x}_1+e{y}_1+j}\left[\begin{array}{c}{a}_1{x}_1+b_1{y}_1+c_1\\ a_2{x}_1+b_2{y}_1+c_2\end{array}\right].$$

$$f_i(\mathbf{x}):=\frac{a_{i}x+b_{i}y+c_i}{dx+ey+j},i=1,2.$$

$$A(\mathbf{x}):=\left[\begin{array}{c}{{\mathbf{\alpha }}_1}^T\mathbf{x}+{\beta }_1\\ {{\mathbf{\alpha }}_2}^T\mathbf{x}+{\beta }_2\end{array}\right],$$

$$\frac{\alpha X+\beta Y+\gamma }{X+\delta }=\alpha +\frac{\beta Y-\alpha \delta +\gamma }{X+\delta }$$

$$H=\frac{1}{(X+\delta {)}^3}\left[\begin{array}{cc}2(\beta Y-\alpha \delta +\gamma )& -\beta (X+\delta )\\ -\beta (X+\delta )& 0\end{array}\right]$$

Full text

Keywords: Optimization, Control, Computer Vision.

1 Introduction

We will work in ${\mathbb{R}}^{k},k\geq 1$ where ${\bf x}\in{{\mathbb{R}}^{k}}$ is the column vector $[x_{1}\,x_{2}\,\cdots\,x_{k}]^{T}$ and $T$ denotes transpose. A function $g:{{\mathbb{R}}^{k}}\rightarrow{\mathbb{R}}$ is an affine function provided there exists ${\bm{\alpha}}\in{{\mathbb{R}}^{k}},\,{\rm and}\,\beta\in{\mathbb{R}}$ such that $g({\bf x})={\bm{\alpha}}^{T}{\bf x}+\beta$.

1.1 Min-Max approximation

In this paper, we are interested in Min-Max approximation to a continuous $f:\Delta\subset\mathbb{R}^{k}\xrightarrow{}\mathbb{R}$ by affine approximants to $f$. See for example D ; Da ; L ; Nik ; Tem . By imposing the restriction $\Delta$ is a simplex with non-empty interior, we obtain a complete characterization of these approximants and an explicit formula for a unique best approximant. Even in the case of an interval, $k=1$, our main result Theorem 2.1 provides an extension of [Da , Corollary 7.6.3]. For $k=1$, our main result also provides an extension of the Chebyshev equioscillation theorem for linear approximants with an explicit unique formula for the best approximant. See Section 4. We show interesting connections of Theorem 2.1 to graphics. See Section 5.

In order to state our results, we need the following notation. For a continuous function $g:\mathbb{R}^{k}\to\mathbb{R}$, and simplex $\Delta$, we adopt the usual convention of:

[TABLE]

We will answer the following

1.2 Problem

Let $f:\Delta\subset\mathbb{R}^{k}\to\mathbb{R}$ be a continuous function where $\Delta$ is a compact domain in $\mathbb{R}^{k}$. Find conditions on $\Delta$ and $f$ which allow for the construction of the Min-Max affine approximation problem (1.1) to $f$ over $\Delta$. Equivalently, find conditions on $\Delta$ and $f$ which allow for the explicit construction of an ${\bm{\alpha}}\in{{\mathbb{R}}^{k}}$ and a $\beta\in{\mathbb{R}}$ which solve

[TABLE]

2 Main result: Theorem 2.1

Our main result is

Theorem 2.1

Let $\{{\bf a}_{1},\dots,{\bf a}_{k+1}\}$ be $k+1$ affinely independent points in $\mathbb{R}^{k}$ so that their convex hull $\Delta={\rm CH}({\bf a}_{1}\dots\bf a_{k+1})$ is a $k$-simplex and assume $f:\Delta\subset\mathbb{R}^{k}\rightarrow\mathbb{R}$ is a continuous convex or concave function over $\Delta$. Then the Min-Max affine approximant to $f$ over $\Delta$ is the hyperplane average $\sigma:=\frac{\pi+\rho}{2}$, where $\pi$ is the affine hyperplane ${\rm AS}(({\bf a}_{1},f({\bf a}_{1}))\dots({\bf a}_{k+1},f({\bf a}_{k+1})))$ in $\mathbb{R}^{k+1}$ and $\rho$ is the supporting hyperplane to the graph of $f$ parallel to $\pi$.

Here ${\rm AS}$ denotes the affine span and any hyperplane $\tau$ is identified with its graph $\{({\bf x},\tau({\bf x}))\in{\mathbb{R}}^{k+1}:x\in{\mathbb{R}}^{k}\}$.

Theorem 2.1 belongs to an interesting class of related optimization problems which can be found for example in ANW ; Br ; Da ; D ; Korn ; L ; Mir ; Nik ; Tem .

We know of no convex domain other than a $k$-simplex where we can generate a hyperplane $\pi$ as in Theorem 2.1 for which we can verify that the graph of $f$ over the domain lies entirely above or entirely below that hyperplane.

We now present two remarks regarding Theorem 2.1.

Remark  2.1 1

The following stronger version of Theorem 2.1 holds which has an identical proof as Theorem 2.1.

Theorem 2.2

Given a continuous function $f$ on a simplex $\Delta$ with the following property: The graph of $f$ lies entirely above or below its secant hyperplane $\pi$ through the graph points of $f$ over the vertices of $\Delta$. Then an expression for the Min-Max affine approximation of $f$ on the simplex $\Delta$ has graph given by the hyperplane $\pi+d$ where 2d is the non-zero extremum value of $f-\pi$ on $\Delta$. An alternative definition of $\pi$ is the affine interpolant to $f$ at the $k+1$ vertices of the simplex $\Delta$.

Remark  2.2 2

In this remark, we speak to a geometric description of the hyperplane average in Theorem 2.1 and write formulae for the optimal ${\bm{\alpha}}$ and $\beta$ in the notation of Problem 1.2, expression (1).

Let the vector ${\bm{\alpha}}^{\prime}$ and the scalar $\beta^{\prime}$ be defined as the solution to the following equation.

[TABLE]

Secondly define:

[TABLE]

Then set ${\bm{\alpha}}:={\bm{\alpha}}^{\prime}$ and $\beta:=\frac{1}{2}(\beta^{\prime}+\beta^{\prime\prime})$. It is easily checked that Problem 1.2, expression (1) is optimized at $({\bm{\alpha}},\beta)$.

We end this remark by saying that the computation of the optimal $\beta$ above costs a minimization of a convex function (a linear shift of $\pm f$) over the set $\Delta$.

3 The proof of Theorem 2.1

The key ideas in our proof of Theorem 2.1 are the two equivalent definitions of a convex function. $f:\Delta\subset\mathbb{R}^{k}\to\mathbb{R}$ is convex provided $\textnormal{for all }\gamma_{i}\geq 0:\Sigma_{i=1}^{k+1}\gamma_{i}=1,$ and any affine independent set of $k+1$ points $\{{{\bf a}}^{\prime}_{1},\ldots,{{\bf a}}^{\prime}_{k+1}\}$, $f(\Sigma_{i=1}^{k+1}\gamma_{i}{{\bf a}}^{\prime}_{i})\leq\Sigma_{i=1}^{k+1}\gamma_{i}f({{\bf a}}^{\prime}_{i})$, i.e. the graph of $f$ over the convex hull of $k+1$ affinely independent points lies below the convex hull of the $k+1$ image points $\{[{{\bf a}}^{\prime}_{i}{}^{T}\,\,f({{\bf a}}^{\prime}_{i})]^{T}:1\leq i\leq k+1\}$, or equivalently, $\textnormal{for each }{\bf x}\in\Delta,$ $\textnormal{and each support plane at }[{\bf x}^{T}\,\,f({\bf x})]^{T}$ $\textnormal{ with slope }{\bf\eta}\in{\mathbb{R}}^{k}$, $f({\bf y})\geq f({\bf x})+{\bf\eta}^{T}({\bf y}-{\bf x}),$ $\textnormal{for all }{\bf y}\in\Delta$, i.e. the graph of $f$ lies on or above any supporting hyperplane.

For our proof of Theorem 2.1, we advise the reader to use Figure 1 for intuition.

Proof

First assume that $f$ is convex, so its graph over $\Delta$ lies on or below the hyperplane $\pi$. Let $-2d:=\inf\{l:\pi+l{\bf e}_{k+1}\cap graph(f)\neq\varnothing\}$. Then any admissible $l$ must satisfy $l\leq 0$ and so this in turn means $d\geq 0$. Here, ${\bf e}_{k+1}$ is the $k+1^{st}$ basis vector in $\mathbb{R}^{k+1}$ so each admissible $l$ gives a downward translate of $\pi$ with non-empty intersection with the graph of $f$. Since $\Delta$ is compact and $f$ is continuous, the $\inf$ is actually a $\min$ and so their exists at least one graph point $(y,f(y))$ on $\rho:=\pi-2d$.

Thus $\rho$ is a supporting hyperplane (tangent plane if $f$ is differentiable at ${\bf y}$). Moreover, at the points ${\bf a}_{i}$ an easy computation shows $\pi-d=(2\pi-2d)/2=(\pi+\rho)/2=\rho+d$. These and the construction of $\sigma$ imply $f({\bf a}_{i})-\sigma({\bf a}_{i})=d=\sigma(y)-f(y)$. We also observe that $\forall z\in\Delta$, $|f(z)-\sigma(z)|\leq\sigma(y)-f(y)=d$, as $f(z)$ is between $\pi$ and $\rho$ whose midplane is $\sigma$. Now with the above in hand we are able to argue as follows. Assume $\mu$ is a best affine approximating plane. Then first $\mu({\bf a}_{i})\geq\sigma({\bf a}_{i}),~{}~{}i\in[1\dots k+1]$ otherwise the maximal distance increases. On the other hand, writing $y=\sum_{i=1}^{k+1}\gamma_{i}{\bf a}_{i}$ for constants $\gamma_{i}\geq 0:\Sigma_{i=1}^{k+1}\gamma_{i}=1$, we have $\mu(y)=\sum_{i=1}^{k+1}\gamma_{i}\mu({\bf a}_{i})\leq\sigma(y)$ otherwise the maximal distance increases. We deduce that $\mu=\sigma$. This concludes the proof of Theorem 2.1 in case $f$ is convex.

In the case that $f$ is concave, since $\pi={\rm AS}((a_{1},f({\bf a}_{1}))\dots({\bf a}_{k+1},f({\bf a}_{k+1})))$ and the graph of $f$ over $\Delta$ lies on or above $\pi$, we can repeat the convex argument replacing $d\text{ by }-d$, $\inf\text{ by }\sup$ and interchanging $\leq\text{ and }\geq$. Alternatively note $-f$ is convex and one checks that the negative of the solution for $-f$ is just $\frac{\pi+\rho}{2}$.

Note that even if $f$ is not convex (or concave) and not even smooth the method produces a best uniform approximation for the $k+2$ points comprised of the $k+1$ points $({\bf a}_{1},f({\bf a}_{1}))\ldots,({\bf a}_{k+1},f({\bf a}_{k+1}))$ and $({\bf y},f({\bf y}))$.

4 Theorem 2.1: The case $k=1$.

For $k=1$, our main result Theorem 2.1 provides an extension of the classical Chebyshev equioscillation theorem, see Theorem 4.1 below, for linear approximants with an explicit unique formula for the best approximant.

We now explain this.

4.1 Chebyshev systems and Chebyshev-Markov equioscillation

We will work with the real interval $[p,q]$ where $p<q$ and the space of real valued continuous functions $f:[p,q]\to\mathbb{R}$. As per convention, we denote this space by $C([p,q])$.

A set of $l+1,\,l\geq 0$ functions $\left\{u_{j}(t)\right\}_{j=0}^{l}$, $u_{j}\in C([p,q])$ is called a Chebyshev (Haar) system on the interval $[p,q]$ if any linear combination

[TABLE]

with not all coefficients $c_{j}$ zero, has at most $l$ distinct zeros in $[p,q]$. 111Alfred Haar, 1885-1933, was a Hungarian mathematician. In 1904 he began to study at the University of Göttingen. His doctorate was supervised by David Hilbert.

The $l+1$ dimensional subspace $U_{l}\subset C([p,q)])$ spanned by a Chebyshev system $\left\{u_{j}(t)\right\}_{j=0}^{l}$ on $[p,q]$ defined by

[TABLE]

is called a Chebyshev space on $[p,q]$.

The classical Chebyshev equioscillation theorem, see for example [Kr , Chapter 9, Theorem 4.4] and Sch is the following:

Theorem 4.1

Let $f\in C([p,q])$ and let $U_{l}$ be a $l+1$ Chebyschev space on an interval $[p,q]$. Then $\hat{u}\in U_{l}$ is a best uniform approximant to $f$ on $[p,q]$, that is $u$ satisfies

[TABLE]

if and only there exists $l+2$ points $\left\{x_{1},...,x_{l+2}\right\}$ with $p\leq x_{1}<...<x_{l+2}\leq q$ such that

[TABLE]

Motivated by Theorem 4.1 we have:

4.2 Chebyshev equioscillation

Let $f\in C([p,q)]$. A Chebyshev polynomial $\hat{h}_{l}$ (when it exists) of degree at least $l\geq 1$, is the polynomial which best uniformly approximates $f$ on $[p,q]$, i.e,

2

[TABLE]

where ${\Pi}_{l}$ is the set of polynomials of degree at most $l$.

The following is often called the Chebyshev equioscillation theorem.

Theorem 4.2

Let $f\in C([a,b])$. Then $\hat{h_{l}}$ exists if there exists $l+2$ points $\left\{x_{1},...,x_{l+2}\right\}$ with $p\leq x_{1}<...<x_{l+2}\leq q$ such that

[TABLE]

4.3 The case $l=1$ of Theorem 4.2

We now show how Theorem 4.2 with $l=1$ corresponds to Theorem 2.1 for $k=1$.

We may assume that $p<q$ and for the moment we do not assume anything about $f:[p,q]\to\mathbb{R}$. Define now a linear function $L:[p,q]\to\mathbb{R}$ from $f$ in terms of parameters $d$ and $y$ to be determined later as follows:

[TABLE]

Now since $L(y)-f(y)=d$, if we assume $L(p)-f(p)=-d=L(q)-f(q)$ then

[TABLE]

Thus

[TABLE]

Also,

[TABLE]

So:

[TABLE]

Thus $m=\frac{f(q)-f(p)}{p-q}$ in the case when $f$ or $-f$ is a convex and differentiable function, by the mean value theorem.

It is clear that we have proved Theorem 2.1 once we are able to choose $r$ to maximize $d$ if this is possible for the given $f$. In the case when $f$ is convex, we see that $m=f^{\prime}(r)$. It is clear that the argument works when $f$ is concave, which implies $-f$ is convex,

In the case of $k=1$, we could write

[TABLE]

[TABLE]

which is either an upward or downward hyperbola for which the secant between $[p,q]$ lies above (or below) the curve. Then following the ”mean value ” argument, leads to $r$ with derivative =slope and via our visualization leads to a line with the same slope but halfway between the secant and tangent to $r$.

5 Connections of Theorem 2.1 to graphics

One consequence of Theorem 2.1 gives an interesting connection to graphics. We provide our ideas below.

We are given a flat object $\mathcal{O}$ and want to render it from a given perspective, camera setup. The resulting image is a projective (AKA perspective or homography) transformation of $\mathcal{O}$, $P(\mathcal{O})$. Thus, each pixel (color) in $\mathcal{O}$, (an ordered pair $(x_{1},y_{1})$ in $\mathbb{R}^{2}$) is transformed to a new pixel location (point) 2 via the action of $P$

[TABLE]

Practically, to render this object which can have 10’s of millions of pixels it is useful to have a good fast approximation of the transformation not entailing division which can cause numerical instabilities. Thus, we seek affine approximants to $P$. One known method is to simply take the affine approximant to be the linear terms of the Taylor expansion around one point, the tangent approximation.

We aim to provide a better 2d-affine approximant to $P$ than the tangent approximation. From Equation 5.1 the components of $P({\bf x})$ are given by

[TABLE]

From our main Theorem 2.1 we know that if we can find a triangle, $\Delta$, containing (a large part of) $\mathcal{O}$ such that each $f_{i}({\bf x})$ is convex or concave then there exist Min-Max affine approximations ${{\bm{\alpha}}_{i}}^{T}{\bf x}+\beta_{i}$ to each $f_{i}$. Then forming the affine transformation $A$ of 2-space given by

[TABLE]

provides, component wise, the best uniform affine approximants on $\Delta$.

Since a differentiable function is convex or concave on a domain exactly when it’s Hessian is respectively positive or negative semidefinite the following is a key tool for the application of our main result.

Proposition 5.1 5.1

A symmetric $2x2$ matrix $D=\begin{bmatrix}p_{11}&p_{12}\\ p_{12}&p_{22}\end{bmatrix}$ is * positive or negative semidefinite according as $signum(p_{11})={\pm}\textnormal{signum }(p_{11}p_{22}-p_{12}^{2}).$*

The Hessian of

[TABLE]

is

[TABLE]

with determinant $\frac{-\beta^{2}}{(X+\delta)^{4}}$. The sign of $H[1,1]=2(\beta Y-\alpha\delta+\gamma)$ depends on $Y\begin{matrix}<\

\end{matrix}\frac{\alpha\delta-\gamma}{\beta}$.

When we combine Proposition 5.1 with the Hessian of Eq. 5.4 we see that $\frac{\alpha X+\beta Y+\gamma}{X+\delta}$ in each of the connected components of ${\mathbb{R}}^{2}\setminus\{\{X=-\delta\}\cup\{Y=\frac{\alpha\delta-\gamma}{\beta}\}\}$ is either convex or concave. With the 2 coordinate functions of the projective transformation ($X$ and $Y$) we end with 6 regions in ${\mathbb{R}}^{2}$ where Theorem 2.1 holds, Figure 2.

In order to transform a general projective transformation as in Eq.  5.3 to the form in Eq. 5.4, with a simple denominator, we rotate the axes so that $X$ is in the direction of $\begin{bmatrix}d\\ e\end{bmatrix}$ and normalize so that the coefficient of $X$ in the denominator is 1.

In Figure 3 we show a few examples of the various affine approximations $A$ of projective transformations $P$ using this idea.

The first column is the original image $\mathcal{O}$, the second column is the image transformed by a projective transformation $P$, modelling a new viewpoint, the third column is the transformation based on the affine approximation $A$ of $P$ using Theorem 2.1 and a user defined triangle while the fourth column is computed using a Taylor series approximation of $P$ around the image’s center. The images in column 3 should visually look closer to those in column 2 than those in column 4, the Taylor approximation.