On local analysis

TL;DR

This paper extends smoothed analysis estimates for condition numbers to Gaussian distributions and introduces a local analysis concept to study their behavior around specific points.

Contribution

It provides a Gaussian extension of smoothed analysis estimates and introduces a new local analysis framework for condition numbers.

Findings

Extended smoothed analysis estimates to Gaussian distributions.

Introduced a local analysis notion for condition numbers.

Captured behavior of condition numbers around specific points.

Abstract

We extend to Gaussian distributions a result providing smoothed analysis estimates for condition numbers given as relativized distances to illposedness. We also introduce a notion of local analysis meant to capture the behavior of these condition numbers around a point.

Equations

$$\max_{\overline{A}\in\mathbb{S}(\mathbb{R}^{n\times n})}\mathop{\mathbb{E}}_{A\sim N(\overline{A},\sigma^{2}{\rm Id})}\ln\kappa(A)\leq\ln\Big{(}\frac{n}{\min\{\sigma,1\}}\Big{)}+{\cal O}(1),$$

$$\mathcal{C}(x)=\frac{\Vert x\Vert }{d(x,\Sigma )},$$

$$\underset{\overline{x}\in {\mathbb{S}}^N}{\max }{\mathbb{E}}_{x\in B_{\mathbb{S}}(\overline{x},\theta )}\ln \mathcal{C}(x)\le \ln \frac{Nd}{\sin \theta }+2(\ln 2+1)$$

$$\underset{\overline{x}\in {\mathbb{S}}^N}{\max }{\mathbb{E}}_{x\sim N(\overline{x},{\sigma }^2\mathrm{Id})}\ln \mathcal{C}(x)\le H(N,d,\sigma )$$

$$\underset{\sigma \to 0}{\lim }H(N,d,\sigma )=\infty \text{mbox}and\underset{\sigma \to \infty }{\lim }H(N,d,\sigma )=\ln (Nd)+\mathcal{O}(1).$$

$${\mathbb{E}}_{x\sim \mathcal{D}(\overline{x})}\ln \mathcal{C}(x)$$

$$B_{\mathbb{S}}(\overline{x},\theta )=\{x\in {\mathbb{S}}^N:0\le \sphericalangle (x,\overline{x})\le \theta \}=\{x\in {\mathbb{S}}^N:⟨x,\overline{x}⟩\ge \cos \theta \}$$

$${\mathcal{O}}_N=\frac{2{\pi }^{\frac{N+1}{2}}}{\Gamma (\frac{N+1}{2})}$$

$$\mathsf{vol}(B(0,1))=\frac{{\mathcal{O}}_N}{N+1}$$

$$\frac{{\mathcal{O}}_N}{\sqrt{2\pi (N+1)}}(\sin \theta {)}^N\le \mathsf{vol}(B_{\mathbb{S}}(x,\theta ))\le \frac{{\mathcal{O}}_N}{2}(\sin \theta {)}^N.$$

$$\mathcal{C}:{\mathbb{R}}^{N+1}\to [1,\infty ],\mathcal{C}(x)=\frac{\Vert x\Vert }{d(x,\Sigma )},$$

$$\mathcal{C}(x)=\frac{1}{d_{\sin }(x,\Sigma \cap {\mathbb{S}}^N)}.$$

$${\mathbb{E}}_{x\in B_{\mathbb{S}}(\overline{x},\theta )}\ln \mathcal{C}(x)={\mathbb{E}}_{x\in B_{\sin }(\overline{x},\rho )}\ln \mathcal{C}(x)\le \ln \frac{Nd}{\sin \theta }+K$$

$${\mathbb{E}}_{x\in {\mathbb{S}}^N}\ln \mathcal{C}(x)\le \ln (Nd)+K,$$

$$\mathop{\mathbb{E}}_{x\in B_{\mathbb{S}}(\overline{x},\theta)}\ln\mathop{\mathscr{C}}(x)\leq\left\{\begin{array}[]{ll}\ln\dfrac{Nd}{\rho+\frac{1-\rho}{\mathop{\mathscr{C}}(\overline{x})}}+\ln 12+2&\mbox{if $\rho>\dfrac{1}{2\mathop{\mathscr{C}}(\overline{x})+1}$}\\ \ln\dfrac{1}{\rho+\frac{1-\rho}{\mathop{\mathscr{C}}(\overline{x})}}+\ln 4&\mbox{if $\rho\leq\dfrac{1}{2\mathop{\mathscr{C}}(\overline{x})+1}$.}\end{array}\right.$$

$${\mathbb{E}}_{x\in B_{\mathbb{S}}(\overline{x},\theta )}\ln \mathcal{C}(x)\le H(N,d,\theta ,\mathcal{C}(\overline{x})).$$

$$\rho (2\mathcal{C}(\overline{x})+1)\ge 1⟺2\rho \mathcal{C}(\overline{x})+\rho \ge 1⟺2\rho \mathcal{C}(\overline{x})\ge 1-\rho ⟺2\rho \ge \frac{1-\rho }{\mathcal{C}(\overline{x})}$$

$$\rho=\frac{1}{3}\rho+\frac{1}{3}(2\rho)\geq\frac{1}{3}\Big{(}\rho+\frac{1-\rho}{\mathop{\mathscr{C}}(\overline{x})}\Big{)},\quad\mbox{i.e}\quad\frac{1}{\rho}\leq\frac{3}{\rho+\frac{1-\rho}{\mathop{\mathscr{C}}(\overline{x})}}.$$

$${\mathbb{E}}_{x\in B_{\mathbb{S}}(\overline{x},\theta )}\ln \mathcal{C}(x)$$

$$\le \ln \frac{3Nd}{\rho +\frac{1-\rho }{\mathcal{C}(\overline{x})}}+\ln 4+2=\ln \frac{Nd}{\rho +\frac{1-\rho }{\mathcal{C}(\overline{x})}}+\ln 12+2.$$

$$\frac{1}{2\mathop{\mathscr{C}}(\overline{x})+1}=\frac{1}{4}\Big{(}\frac{1}{2\mathop{\mathscr{C}}(\overline{x})+1}+\frac{3}{2\mathop{\mathscr{C}}(\overline{x})+1}\Big{)}>\frac{1}{4}\Big{(}\rho+\frac{1-\rho}{\mathop{\mathscr{C}}(\overline{x})}\Big{)}$$

$$2\mathcal{C}(\overline{x})+1<\frac{4}{\rho +\frac{1-\rho }{\mathcal{C}(\overline{x})}}.$$

$$\frac{1}{\mathcal{C}(x)}$$

$$\ge \frac{1}{\mathcal{C}(\overline{x})+\frac{1}{2}}-\frac{1}{2\mathcal{C}(\overline{x})+1}=\frac{1}{2\mathcal{C}(\overline{x})+1},$$

$$\mathcal{C}(x)\le 2\mathcal{C}(\overline{x})+1<\frac{4}{\rho +\frac{1-\rho }{\mathcal{C}(\overline{x})}}$$

$$\ln \mathcal{C}(x)\le \ln \frac{1}{\rho +\frac{1-\rho }{\mathcal{C}(\overline{x})}}+\ln 4.$$

$$\rho \mapsto 2(Nd-1){\rho }^{{\log }_{{}_{\frac{1}{2\mathcal{C}(\overline{x})}}}\frac{1}{2}}+1$$

$$\varphi\Big{(}\frac{1}{2\mathop{\mathscr{C}}(\overline{x})+1}\Big{)}\leq\varphi\Big{(}\frac{1}{2\mathop{\mathscr{C}}(\overline{x})}\Big{)}=2(Nd-1)\frac{1}{2}+1=Nd.$$

$$\ln \frac{1}{\rho +\frac{1-\rho }{\mathcal{C}(\overline{x})}}+\ln 4\le \ln \frac{\phi (\rho )}{\rho +\frac{1-\rho }{\mathcal{C}(\overline{x})}}+\ln 12+2\text{mbox}for0\le \rho <\frac{1}{2\mathcal{C}(\overline{x})+1}$$

$$\ln \frac{Nd}{\rho +\frac{1-\rho }{\mathcal{C}(\overline{x})}}+\ln 12+2\le \ln \frac{\phi (\rho )}{\rho +\frac{1-\rho }{\mathcal{C}(\overline{x})}}+\ln 12+2\text{mbox}for\frac{1}{2\mathcal{C}(\overline{x})+1}\le \rho \le 1,$$

$${\mathbb{E}}_{x\in B_{\mathbb{S}}(\overline{x},\theta )}\ln \mathcal{C}(x)\le \ln \frac{\phi (\rho )}{\rho +\frac{1-\rho }{\mathcal{C}(\overline{x})}}+\ln 12+2.$$

Full text

On local analysis

Felipe Cucker

Dept. of Mathematics

City University of Hong Kong

macucker@cityu.edu.hk Partially supported by a GRF grant from the Research Grants Council of the Hong Kong SAR (project number CityU 11302418).   

Teresa Krick

Departamento de Matemática & IMAS

Univ. de Buenos Aires & CONICET

ARGENTINA

krick@dm.uba.ar Corresponding author. Partially supported by grant CONICET-PIP2014-2016-112 20130100073CO.

**Abstract. We extend to Gaussian distributions a result providing smoothed analysis estimates for condition numbers given as relativized distances to ill-posedness. We also introduce a notion of local analysis meant to capture the behavior of these condition numbers around a point.

2010 Mathematics Subject Classification: Primary 65Y20, Secondary 65F35.

Keywords: Conic condition number. Smoothed analysis. Local analysis.**

1 Introduction

In the 1990s D. Spielman and S.H. Teng introduced the notion of smoothed analysis, in an attempt to give a more realistic analysis of the practical performance of an algorithm than those obtained through the use of worst-case or average-case analyses. In a nutshell, this new paradigm in probabilistic analysis interpolates between worst-case and average-case by considering the worst-case (over the data) of the average value (over possible random perturbations) of the analyzed quantity. See, for instance, [7] for an overview.

An example of this analysis to the quantity $\ln\kappa(A)$, where $A$ is a square matrix and $\kappa(A):=\|A\|\,\|A^{-1}\|$, was provided by M. Wschebor in [10]. Wschebor showed that

[TABLE]

where here, and in what follows, $x\sim N(\overline{x},\sigma^{2}{\rm Id})$ indicates that $x$ is drawn from an isotropic Gaussian distribution centered at $\overline{x}$ with covariance matrix $\sigma^{2}{\rm Id}$. The behavior of the bound $H(n,\sigma)$ in the right-hand side of (1) shows two expected properties of a smoothed analysis:

(SA1)

When $\sigma\to 0$, $H(n,\sigma)$ tends to its worst-case value (there are no random perturbations of the input in this case).

(SA2)

When $\sigma\to\infty$, $H(n,\sigma)$ tends to the average value of the analyzed quantity (the random perturbation is over all the input data in this case).

Indeed, the convergence of $H(n,\sigma)$ to infinity when $\sigma\to 0$ is clear, and with it (SA1). And a result of A. Edelman [6] proves that $\mathop{\mathbb{E}}_{A\sim N(0,\sigma^{2}{\rm Id})}\ln\kappa(A)=\ln n+{\cal O}(1)$, thus showing (SA2).

The main agenda of this paper is to introduce the notion of local analysis, which aims to study locally at a base point $\overline{x}$ the average value over possible random perturbations of the analyzed quantity, without taking then the worst-case over all input data. The benefit of such analysis is that it provides information depending directly on the base point instead of assuming a worst-case, as in the smoothed analysis.

We illustrate this notion by developing it for a conic condition number. This is a condition number satisfying a Condition Number Theorem. We next describe more precisely this notion and its context.

In 1936 Eckart and Young [5] proved that for a square matrix $A$, $\kappa(A)=\|A\|/d(A,\Sigma)$ where $\Sigma$ is the set of non-invertible matrices and $d$ denotes distance. This result came to be known as the Condition Number Theorem, even though it was proved more than ten years before the introduction of condition numbers by Turing [8] and von Neumann and Goldstine [9]. In 1987 J. Demmel observed (and proved) that similar Condition Number Theorems hold true for the condition numbers of various problems [3]. More precisely, he showed that these condition numbers were either equal to or closely bounded by the (normalized) inverse to the distance to ill-posedness. That is, that for an input data $x$ of the problem at hand, the condition number of $x$ for that problem is either equal to or closely bounded by

[TABLE]

where $\Sigma\neq\{0\}$ is an algebraic cone of ill-posed inputs. One year later, Demmel [4] derived general average analysis bounds for those (conic) condition numbers. These bounds depend only on the dimension $N+1$ of the ambient space, the codimension of $\Sigma$, and its degree. He carried out this idea for the complex case and stated it for the real case (requiring $\Sigma$ to be complete intersection) based on an unpublished (and not findable anywhere) result by Ocneanu. The underlying probability distribution is the isotropic Gaussian on $\mathbb{R}^{N+1}$ but it is easy to observe that the bounds hold as well for the uniform distribution on the unit sphere $\mathbb{S}^{N}$ (or, equivalently, on any half-sphere, due to the equality $\mathop{\mathscr{C}}(-x)=\mathop{\mathscr{C}}(x)$).

In [2] Demmel’s idea was extended to perform a smoothed analysis of the conic condition number $\mathop{\mathscr{C}}(x)$ in the case that $\Sigma$ is the zero set of a single real homogeneous polynomial $F$ in $N+1$ variables. For this analysis one considers the centers $\overline{x}$ of the distributions in $\mathbb{S}^{N}$ (as in (1)) and there are two natural choices for the distribution itself: a Gaussian supported in $\mathbb{R}^{N+1}$ or a uniform on a spherical cap in $\mathbb{S}^{N}$. The uniform case is studied in [2], where the following bound is obtained for $\theta\in[0,\pi/2]$:

[TABLE]

where $d$ is the degree of $F$ and $B_{\mathbb{S}}(\overline{x},\theta)$ is the spherical cap of radius $\theta$ centered at $\overline{x}$ which we endow with the uniform distribution. This bound $H(N,d,\theta)$ recovers an average analysis in the particular case that the spherical cap is a half-sphere. That is,

(SA2’)

$H(N,d,\pi/2)=\ln(Nd)+{\cal O}(1)$, is the average value of $\ln\mathop{\mathscr{C}}(x)$ for $x\in\mathbb{S}^{N}$, see [4].

A smoothed analysis of the conic condition number $\mathop{\mathscr{C}}(x)$ in the Gaussian case $N(\overline{x},\sigma^{2}{\rm Id})$ was still lacking, and it is one of the results we present in this paper, since it is strongly linked with our local analysis as we will see below. Theorem 4.1 shows that

[TABLE]

where $H(N,d,\sigma)$ is an explicit bound that satisfies (SA1) and (SA2). That is

[TABLE]

With respect to local analysis, the gist is to obtain bounds for the quantities

[TABLE]

where $\overline{x}\in\mathbb{S}^{N}$ and ${\mathcal{D}}(\overline{x})$ is either the uniform distribution on the spherical cap $B_{\mathbb{S}}(\overline{x},\theta)$ or the Gaussian $N(\overline{x},\sigma^{2}{\rm Id})$.

These bounds will be expressions $H(N,d,\nu,\mathop{\mathscr{C}}(\overline{x}))$ where $\nu$ is either $\theta$ or $\sigma$ depending on the underlying distribution, which should coincide with smoothed analysis bounds when $\mathop{\mathscr{C}}(\overline{x})=\infty$. More precisely, if we denote by $H_{\infty}(N,d,\nu)$ the result of replacing $\mathop{\mathscr{C}}(\overline{x})$ by $\infty$ in $H(N,d,\nu,\mathop{\mathscr{C}}(\overline{x}))$ then we want the following:

(LA0)

$H_{\infty}(N,d,\nu)$ has the same behavior as the smoothed analysis bound $H(N,d,\nu)$.

Furthermore, when $\mathop{\mathscr{C}}(\overline{x})<\infty$ we seek the following limiting behavior:

(LA1)

$\displaystyle\lim_{\nu\to 0}H(N,d,\nu,\mathop{\mathscr{C}}(\overline{x}))=\ln(\mathop{\mathscr{C}}(\overline{x}))+{\cal O}(1)$, the local complexity at $\overline{x}$.

(LA2)

$\displaystyle\lim_{\sigma\to\infty}H(N,d,\sigma,\mathop{\mathscr{C}}(\overline{x}))=\ln(Nd)+{\cal O}(1)$ in the Gaussian case, the average complexity.

(LA2’)

$H(N,d,\pi/2,\mathop{\mathscr{C}}(\overline{x}))=\ln(Nd)+{\cal O}(1)$ in the uniform case, the average complexity.

Indeed, we show that this is the case in Theorem 3.1 (uniform case) and Theorem 4.8 (Gaussian case).

Acknowledgments. We are grateful to Pierre Lairez for many useful discussions. In particular, for pointing to us an argument in Proposition 4.2.

2 Notations and preliminaries

In all what follows we consider the space $\mathbb{R}^{N+1}$ endowed with the standard inner product $\langle~{},~{}\rangle$ and its induced norm $\|~{}\|$. Within this space we have the unit sphere $\mathbb{S}^{N}=\{x\in\mathbb{R}^{N+1}:\,\|x\|=1\}$, and for $\overline{x}\in\mathbb{S}^{N}$ we denote by $B(\overline{x},r)=\{x\in\mathbb{R}^{N+1}:\,\|x-\overline{x}\|\leq r\}$ the closed ball centered at $\overline{x}\in\mathbb{R}^{N+1}$ with radius $r\geq 0$, and by

[TABLE]

the spherical cap in $\mathbb{S}^{N}$ centered at $\overline{x}\in\mathbb{S}^{N}$ with radius $0\leq\theta\leq\pi$, that is the closed ball of radius $\theta$ around $\overline{x}$ in $\mathbb{S}^{N}$ with respect to the Riemannian distance in $\mathbb{S}^{N}$.

We will also refer to the sine distance $d_{\sin}$ in $\mathbb{R}^{N+1}\setminus\{0\}$ given by $d_{\sin}(x,\overline{x}):=\sin(\sphericalangle(x,\overline{x}))$. Let $B_{\sin}(\overline{x},\rho):=\{x\in\mathbb{S}^{N}:\,d_{\sin}(x,\overline{x})\leq\rho\}$ denote the closed ball of radius $\rho$ with respect to $d_{\sin}$ around $\overline{x}\in\mathbb{S}^{N}$. This is the union of $B_{\mathbb{S}}(\overline{x},\theta)$ with $B_{\mathbb{S}}(-\overline{x},\theta)$ where $\theta\in[0,\pi/2]$ is such that $\rho=\sin\theta$.

We will denote by ${\cal O}_{N}=\mathsf{vol}(\mathbb{S}^{N})$ the volume of $\mathbb{S}^{N}$. We recall (see [1, Prop. 2.19(a)]) that

[TABLE]

as well as [1, Cor. 2.20]

[TABLE]

and, for $x\in\mathbb{S}^{N}$ and $\theta\in[0,\frac{\pi}{2}]$, the bound (see [1, Lem. 2.34])

[TABLE]

The main object in this paper is a conic condition number on $\mathbb{R}^{N+1}$, i.e. a function given by

[TABLE]

where $\Sigma\neq\{0\}$ is the set of ill-posed inputs in $\mathbb{R}^{N+1}$, which we assume closed under scalar multiplication. We note that $\mathop{\mathscr{C}}(x)\geq 1$ for all $x$ since $0\in\Sigma$. As $\mathop{\mathscr{C}}$ is scale invariant we may restrict to data $x$ lying in $\mathbb{S}^{N}$ where $\mathop{\mathscr{C}}$ can also be expressed as

[TABLE]

3 The uniform case

We endow $B_{\sin}(\overline{x},\rho)$ with the uniform probability measure. A smoothed analysis for this measure is given in [1, Th. 21.1]. Assume that $\Sigma$ is contained in a real algebraic hypersurface, given as the zero set of a homogeneous polynomial of degree $d$. Then, for all $\theta\in[0,\frac{\pi}{2}]$ and $\rho:=\sin\theta$, we have

[TABLE]

and

[TABLE]

where $K=2(\ln 2+1)$. Here $\ln$ denotes Neperian logarithm. We observe that the equality above is due to the fact that $\mathop{\mathscr{C}}(x)=\mathop{\mathscr{C}}(-x)$ for all $x\in\mathbb{S}^{N}$ and that $\mathsf{vol}B_{\sin}(\overline{x},\rho)=\mathsf{vol}B_{\mathbb{S}}(\overline{x},\theta)+\mathsf{vol}B_{\mathbb{S}}(-\overline{x},\theta)$.

The same observation applies to the following result.

Theorem 3.1**.**

Let $\mathop{\mathscr{C}}$ ba a conic condition number on $\mathbb{R}^{N+1}$ with set of ill-posed inputs $\Sigma$. Assume that $\Sigma$ is contained in a real algebraic hypersurface, given as the zero set of a homogeneous polynomial of degree $d$. Let $\overline{x}\in\mathbb{S}^{N}$ and $0\leq\theta\leq\pi$. Then, for $\rho:=\sin\theta$,

[TABLE]

In particular, there is a uniform explicit bound $H(N,d,\theta,\mathop{\mathscr{C}}(\overline{x}))$ –defined in (10) below– such that

[TABLE]

This bound satisfies satisfies (LA0), since $H_{\infty}(N,d,\theta)=\ln\frac{Nd}{\sin\theta}+{\cal O}(1)$ as $H(N,d,\theta)$ in (3), (LA1) and (LA2’).

Proof.  Assume first that $\dfrac{1}{2\mathop{\mathscr{C}}(\overline{x})+1}\leq\rho\leq 1$. In this case, we have

[TABLE]

and we can decompose

[TABLE]

Therefore, by (7),

[TABLE]

We next assume $0\leq\rho<\dfrac{1}{2\mathop{\mathscr{C}}(\overline{x})+1}$. In this case,

[TABLE]

since $\dfrac{3}{2\mathop{\mathscr{C}}(\overline{x})+1}>\dfrac{1}{\mathop{\mathscr{C}}(\overline{x})}>\dfrac{1-\rho}{\mathop{\mathscr{C}}(\overline{x})}$. Equivalently,

[TABLE]

We also use here that for all $x\in B_{\mathbb{S}}(\overline{x},\theta)$,

[TABLE]

and therefore

[TABLE]

which implies

[TABLE]

This shows the first statement. We now derive the expression of a bound $H(N,d,\theta,\mathop{\mathscr{C}}(\overline{x}))$.

Let $\varphi:[0,1]\to\mathbb{R}$ be the function defined by

[TABLE]

where the exponent of $\rho$ in the numerator is the logarithm in base $\dfrac{1}{2\mathop{\mathscr{C}}(\overline{x})}$ of $\dfrac{1}{2}$, which, by continuity, we take to be 0 when $\mathop{\mathscr{C}}(\overline{x})=\infty$. We note that $\varphi$ is concave, monotonically increasing, d satisfies $\varphi(0)=1$, $\varphi(1)=2Nd-1$, and when $\mathop{\mathscr{C}}(\overline{x})=\infty$, $\varphi(\rho)=2Nd-1$. Moreover, by monotonicity,

[TABLE]

This implies, since

[TABLE]

and using also concavity,

[TABLE]

that

[TABLE]

That is,

[TABLE]

Finally, it is trivial to verify, from the specific values taken by $\varphi$ mentioned previously, that $H(N,d,\theta,\mathop{\mathscr{C}}(\overline{x}))$ satisfies (LA0), (LA1) and (LA2’). ∎

4 The Gaussian case

We keep the same conic condition number $\mathop{\mathscr{C}}$ but now consider a Gaussian measure $N(\overline{x},\sigma^{2}{\rm Id})$ in $\mathbb{R}^{N+1}$ centered at $\overline{x}\in\mathbb{S}^{N}$ and with covariance matrix $\sigma^{2}{\rm Id}$ for $0<\sigma<\infty$, that is with density function given by

[TABLE]

Since our local analysis will rely on a smoothed analysis in this case, which is not yet known, we begin by studying a general smoothed analysis for the Gaussian case.

4.1 Smoothed analysis

Let $\overline{x}\in\mathbb{S}^{N}$. We recall that, for any $0\leq\theta\leq\frac{\pi}{2}$,

[TABLE]

and in the particular case $\theta=\frac{\pi}{2}$ we denote

[TABLE]

the open half-sphere centered at $\overline{x}$.

The main result of this section is the following smoothed analysis for the Gaussian distribution.

Theorem 4.1**.**

Let $\mathop{\mathscr{C}}$ be a conic condition number on $\mathbb{R}^{N+1}$ with set of ill-posed inputs $\Sigma$. Assume that $\Sigma$ is contained in a real algebraic hypersurface, given as the zero set of a homogeneous polynomial of degree $d$, and that $N\geq 5$. Then, there exists an explicit bound $H(N,d,\sigma)$ –defined in (13)– such that

[TABLE]

This bound satisfies

(SA1)

$\displaystyle\lim_{\sigma\to 0}H(N,d,\sigma)=\infty$, the worst-case value.

(SA2)

$\displaystyle\lim_{\sigma\to\infty}H(N,d,\sigma)=\ln(Nd)+2(\ln 2+1)$, the average value, in remarkable coincidence with (8).

The following map plays a central role in all what follows,

[TABLE]

The main stepping stone towards the proof of Theorem 4.1 is the following.

Proposition 4.2**.**

Let $\overline{x}\in\mathbb{S}^{N}$. There exists a probability density $f:[0,\frac{\pi}{2}]\to\mathbb{R}_{\geq 0}$ of a random variable $\theta\in[0,\frac{\pi}{2}]$, associated to $\overline{x}$, $\sigma$ and $N$, such that for all measurable function $F:\mathbb{R}^{N+1}\to\mathbb{R}_{\geq 0}$ satisfying $F(x)=F(\lambda x)$ for all $\lambda\in\mathbb{R}^{\times}$, one has

[TABLE]

We begin by proving the following lemma.

Lemma 4.3**.**

For any measurable function $F:\mathbb{R}^{N+1}\to\mathbb{R}_{+}$ satisfying $F(\lambda y)=F(y),\ \forall\lambda\in\mathbb{R}^{\times}$, one has

[TABLE]

where $G_{\overline{x}}:[0,\frac{\pi}{2}]\to\mathbb{R}_{>0}$ is a decreasing function of $\alpha$ defined by

[TABLE]

Proof.  We have

[TABLE]

where the second equality follows from the transformation formula [1, Thm. 2.1] applied to the diffeomorphism

[TABLE]

and

[TABLE]

does not depend on $F$. Now, for $\overline{x},x\in\mathbb{S}^{N}_{+}(\overline{x})$,

[TABLE]

Therefore, $G(x)=:G_{\overline{x}}(\sphericalangle(x,\overline{x}))$ where for $0\leq\alpha\leq\frac{\pi}{2}$,

[TABLE]

which is a continuously differentiable decreasing function of $\alpha$. ∎

Proof of Proposition 4.2.   By Lemma 4.3,

[TABLE]

Now, by the fundamental Theorem of Calculus for $0<\alpha<\frac{\pi}{2}$,

[TABLE]

Replacing this in (12) and changing the order of integration, we obtain

[TABLE]

Now, since

[TABLE]

we obtain

[TABLE]

We now denote

[TABLE]

which is a non-negative function since $G_{\overline{x}}$ is decreasing, and rewrite the equality above as

[TABLE]

where

[TABLE]

We now prove that $H(N,\sigma)=e^{-\frac{1}{2\sigma^{2}}}$: Changing variables $\nu=\frac{\lambda}{\sigma}$ we have

[TABLE]

To estimate the quantity between the square brackets we use the known equality

[TABLE]

together with (4) to obtain

[TABLE]

Therefore

[TABLE]

This implies, by taking $F=1$, that

[TABLE]

i.e.

[TABLE]

Therefore $f$ is a density on $[0,\frac{\pi}{2}]$, and

[TABLE]

∎

Since $\mathop{\mathscr{C}}(x)=\mathop{\mathscr{C}}(\lambda x)$ for all $\lambda\in\mathbb{R}^{\times}$, we can now focus on $F(x):=\ln\mathop{\mathscr{C}}(x)$.

Proposition 4.4**.**

With the notation in Proposition 4.2, we have

[TABLE]

Proof.  Replacing the expectations in the right-hand side of the equality in Proposition 4.2 by their bound in (7) for $\rho=\sin\theta$ and $\rho=1=\sin\frac{\pi}{2}$, we obtain

[TABLE]

where $K=2(\ln 2+1)$. The result follows from the last equality in Proposition 4.2. ∎

Our next goal is to estimate the right-hand side in Proposition 4.4.

Lemma 4.5**.**

Let $0\leq t\leq\frac{\pi}{4}$. Then

[TABLE]

Proof.  Write

[TABLE]

Since $\frac{1}{\sin\theta}\leq\sqrt{2}$ for $\theta\in[\frac{\pi}{4},\frac{\pi}{2}]$, the second term satisfies

[TABLE]

We analyze the first term. Let A=\{(\theta,r)\in[t,\frac{\pi}{4}]\times[0,\ln\big{(}\frac{1}{\sin t}\big{)}]\,:\,r\leq\ln(\frac{1}{\sin\theta})\}, and $A_{r}=\{\theta\in[t,\frac{\pi}{4}]:r\leq\ln(\frac{1}{\sin\theta})\}$. By Fubini’s Theorem we have both

[TABLE]

and

[TABLE]

since $t\leq\frac{\pi}{4}$ implies \ln\sqrt{2}\leq\ln\big{(}\frac{1}{\sin t}\big{)} and when $r\leq\ln\sqrt{2}$, then $A_{r}=[t,\frac{\pi}{4}]$. Therefore,

[TABLE]

by taking $s=e^{-r}$. Finally,

[TABLE]

since $\int_{t}^{\frac{\pi}{2}}f(\theta)\mathrm{d}\theta\leq 1$. ∎

Lemma 4.6**.**

Assume $N\geq 5$. For all $t\in[0,\frac{\pi}{4}]$, one has

[TABLE]

Proof.  For $t\leq\frac{\pi}{2}$,

[TABLE]

for $\Psi$ defined in (11). The first inequality holds because for $\theta\leq t$, $\sphericalangle(x,\overline{x})\leq\theta$ implies $\sphericalangle(x,\overline{x})\leq t$, and the second by Proposition 4.2 applied to F=\mathsf{1l}_{\big{\{}\sphericalangle(\Psi(y),\overline{x})\leq t\big{\}}}. It is then enough to bound the right-hand expression.

We observe that for $0\leq t\leq\frac{\pi}{2}$, the set K=\big{\{}y\in\mathbb{R}^{N+1}\,:\,\sphericalangle(\Psi(y),\overline{x})\leq t\big{\}} is a pointed cone with vertex at [math], central axis passing through $\overline{x}$ and angular opening $\alpha:=2t$. In addition, one can prove by the cosine theorem that this cone is included in the union of the pointed cone $\overline{K}$ with vertex at $\overline{x}$, central axis passing through $2\overline{x}$ and angular opening $2\alpha$ with the intersection $K\cap B(\overline{x},1)$ (see Figure 1). Hence, the measure of $K$ (with respect to $N(\overline{x},\sigma^{2}{\rm Id})$) is bounded by the sum of the measures of $\overline{K}$ and $K\cap B(\overline{x},1)$.

As the vertex $\overline{x}$ of $\overline{K}$ coincides with the center of $N(\overline{x},\sigma)$, the measure of $\overline{K}$ with respect to $N(\overline{x},\sigma)$ equals the proportion of the volume (in $\mathbb{S}(\overline{x},1)$) of the intersection of $\overline{K}$ with $\mathbb{S}(\overline{x},1)$ within this sphere. That is, the measure of $\overline{K}$ with respect to $N(\overline{x},\sigma)$ satisfies

[TABLE]

where, we recall, ${\cal O}_{N}:=\mathsf{vol}(\mathbb{S}^{N})$. Using (6) we deduce that, for $t\in[0,\frac{\pi}{4}]$,

[TABLE]

Also,

[TABLE]

Here we used the well-known lower bound \Gamma(\frac{N+1}{2})>\sqrt{2\pi}\Big{(}\frac{N-1}{2}\Big{)}^{\frac{N}{2}}e^{-\frac{N-1}{2}} (see for instance [1, Eq. 2.14]) for the last inequality. We finish the proof by noting that it can be easily proven by induction, using for instance that $N^{N+1}\geq 2N(N-1)^{N}$, that for all $N\geq 5$, we have

[TABLE]

Lemma 4.7**.**

Assume $N\geq 5$. Then,

[TABLE]

Proof.  We have by Lemma 4.5 with $t=0$,

[TABLE]

where by Lemma 4.6, since $0\leq\arcsin s\leq\frac{\pi}{4}$ for $0\leq s\leq\frac{\sqrt{2}}{2}$,

[TABLE]

We have

[TABLE]

where $c(N,\sigma):=\sqrt[N]{\dfrac{(1-e^{-\frac{1}{2\sigma^{2}}})\sigma^{N+1}}{1+2^{N-1}\sigma^{N+1}}}$. In addition we observe that for all $N\geq 2$, $c(N,\sigma)<\dfrac{\sqrt{2}}{2}$ since

[TABLE]

Rewriting $c(N,\sigma)^{-N}=\dfrac{2^{N-1}+\frac{1}{\sigma^{N+1}}}{1-e^{-\frac{1}{2\sigma^{2}}}}$ we get

[TABLE]

∎

Proof of Theorem 4.1.   By Proposition 4.4 and Lemma 4.7,

[TABLE]

with $K=2(\ln 2+1)$. We then define

[TABLE]

We now verify that $H(N,d,\sigma)$ satisfies (SA1) and (SA2):

(SA1)

\displaystyle\lim_{\sigma\to 0}H(N,d,\sigma)=\displaystyle\lim_{\sigma\to 0}\Big{(}\dfrac{1}{N}\Big{(}1+\ln\big{(}2^{N-1}+\dfrac{1}{\sigma^{N+1}}\big{)}\Big{)}+\ln(Nd)+2(\ln 2+1)\Big{)}

{\ }\qquad\qquad\qquad\quad=\displaystyle\lim_{\sigma\to\infty}\Big{(}\dfrac{N+1}{N}\ln\dfrac{Nd}{\sigma}+{\cal O}(1)\Big{)}\ =\ \infty.

Note that actually the difference of the formula in the last line compared to (7), with the dispersion parameter $\sigma$ replacing $\sin\theta$, is negligible.

(SA2)

$\displaystyle\lim_{\sigma\to\infty}H(N,d,\sigma)=\ln(Nd)+2(\ln 2+1)$, and we recover the well-known, average-case analysis, bound for $\mathop{\mathbb{E}}_{x\in\mathbb{S}^{N}}\ln(\mathop{\mathscr{C}}(x))$ (see [4] and [1, Theorem 21.1]).

∎

4.2 Local analysis

The main result of this section is the following.

Theorem 4.8**.**

Let $\mathop{\mathscr{C}}$ be a conic condition number on $\mathbb{R}^{N+1}$ with $N\geq 6$, with set of ill-posed inputs $\Sigma$. Assume that $\Sigma$ is contained in a real algebraic hypersurface, given as the zero set of a homogeneous polynomial of degree $d$. Let $\overline{x}\in\mathbb{S}^{N}$ and $\sigma\geq 0$. Then, there is an explicit bound $H(N,d,\sigma,\mathop{\mathscr{C}}(\overline{x}))$ –defined in (4.2) below– such that

[TABLE]

This bound satisfies (LA0), (LA1) and (LA2).

In order to prove Theorem 4.8 we need the following lemma.

Lemma 4.9**.**

Assume $N\geq 2$. For all $t\in[0,\pi/2]$,

[TABLE]

Proof.  The idea is to apply Markov’s inequality (e.g. [1, Corollary 2.9]) to the density $f$ to deduce that

[TABLE]

Therefore we need to bound $\displaystyle{\mathop{\mathbb{E}}_{\theta\sim f}(\theta)}$. We first prove that

[TABLE]

where $\Psi$ is given by (11), and then that

[TABLE]

This implies

[TABLE]

To show (14) we apply Proposition 4.2 with $F(y)=\|\Psi(y)-\overline{x}\|$ and get

[TABLE]

We claim that

[TABLE]

Indeed, for $0\leq\alpha:=\sphericalangle(x,\overline{x})\leq\frac{\pi}{2}$, one has

[TABLE]

Therefore, writing $\mathsf{v}(\theta):=\mathsf{vol}(B_{\mathbb{S}}(\overline{x},\theta))$,

[TABLE]

Now, for $0\leq\theta\leq\frac{\pi}{2}$, we have

[TABLE]

which implies

[TABLE]

Using (6) twice we have, for $N\geq 6$,

[TABLE]

and we deduce that $\frac{\mathsf{v}(\theta)-\mathsf{v}(\frac{\theta}{2})}{\mathsf{v}(\theta)}\geq\frac{1}{2}$. With this,

[TABLE]

which shows (17). From (16) and (17) it follows that

[TABLE]

which shows (14). We now show (15). We let $\Psi^{*}(y)$ be the closest point to $\overline{x}$ on the line through [math] and $y$ (see Figure 2) and have

[TABLE]

where the last inequality is a consequence of [1, Prop. 2.10 & Lem. 2.15].

This shows (15). Therefore,

[TABLE]

as desired, and hence,

[TABLE]

Proof of Theorem 4.8.   Let $t:=\arcsin\frac{1}{2\mathop{\mathscr{C}}(\overline{x})}$. Since $\mathop{\mathscr{C}}(\overline{x})\geq 1$, $\frac{1}{2\mathop{\mathscr{C}}(\overline{x})}\leq\frac{1}{2}$ and we have $t\leq\frac{\pi}{6}$. For all $\theta\leq t$ and all $x\in B_{\mathbb{S}}(\overline{x},\theta)$ we have

[TABLE]

which implies $\ln(\mathop{\mathscr{C}}(x))\leq\ln(2\mathop{\mathscr{C}}(\overline{x}))$.

We apply Proposition 4.2 to $F(y)=\ln\mathop{\mathscr{C}}(y)$ and use the previous inequality and the bounds (7) and (8) to obtain

[TABLE]

since

[TABLE]

We next bound each of the first three terms in the right-hand side.

Applying Lemma 4.6 and the inequality $\sin(2t)\leq 2\sin t$ we obtain

[TABLE]

This bounds the first term in (4.2) by

[TABLE]

Second, by Lemma 4.5 since $t\leq\frac{\pi}{6}$, Lemma 4.9 and $t\geq\sin t=\frac{1}{2\mathop{\mathscr{C}}(\overline{x})}$,

[TABLE]

Also, as $t\geq 0$, we have by Lemma 4.7 that

[TABLE]

Putting together this inequality and (4.2) we deduce that the second term in (4.2) is bounded by

[TABLE]

Finally, using again Lemma 4.9 and $t\geq\sin t=\frac{1}{2\mathop{\mathscr{C}}(\overline{x})}$ we obtain

[TABLE]

which bounds the third term in (4.2) by

[TABLE]

Combining (19), (4.2) and (22) with the bound in (4.2), we obtain

[TABLE]

where $\overline{K}=\ln 2+K=3\ln 2+2$. We now verify that $H(N,d,\sigma,\mathop{\mathscr{C}}(\overline{x}))$ satisfies (LA0), (LA1) and (LA2).

(LA0)

When $\mathop{\mathscr{C}}(\overline{x})=\infty$ we get

[TABLE]

which is that of (13) (with a slightly bigger constant) as required in (LA0).

(LA1)

When $\sigma\to 0$, we have

[TABLE]

as required.

(LA2)

Also, when $\sigma\to\infty$, we get

[TABLE]

and we recover the average-case analysis bound for $\mathop{\mathbb{E}}_{x\in\mathbb{S}^{N}}\ln(\mathop{\mathscr{C}}(x))$.

∎