Convexity properties of the condition number

TL;DR

This paper explores the convexity properties of the condition number in matrix spaces and Riemannian manifolds, establishing conditions under which certain functions related to the condition number are convex along geodesics.

Contribution

It introduces a Riemannian structure on matrix spaces and proves convexity of log of the inverse squared smallest singular value, extending to a general notion of self-convexity on manifolds.

Findings

The function A -> log(sigma_n(A)^(-2)) is convex along geodesics when sigma_n(A) has multiplicity 1.

Self-convexity of functions related to distance to submanifolds is characterized by necessary and sufficient conditions.

The square of the condition number is self-convex in projective space and solution varieties.

Abstract

We define in the space of n by m matrices of rank n, n less or equal than m, the condition Riemannian structure as follows: For a given matrix A the tangent space of A is equipped with the Hermitian inner product obtained by multiplying the usual Frobenius inner product by the inverse of the square of the smallest singular value of A denoted sigma_n(A). When this smallest singular value has multiplicity 1, the function A -> log (sigma_n(A)^(-2)) is a convex function with respect to the condition Riemannian structure that is t -> log (sigma_n(A(t))^(-2)) is convex, in the usual sense for any geodesic A(t). In a more abstract setting, a function alpha defined on a Riemannian manifold (M,<,>) is said to be self-convex when log alpha (gamma(t)) is convex for any geodesic in (M,<,>). Necessary and sufficient conditions for self-convexity are given when alpha is C^2. When alpha(x) = d(x,N)^(-2) where d(x,N) is the distance from x to a C^2 submanifold N of R^j we prove that alpha is self-convex when restricted to the largest open set of points x where there is a unique closest point in N to x. We also show, using this more general notion, that the square of the condition number ||A|||_F / sigma_n(A) is self-convex in projective space and the solution variety.