A linearly convergent Gauss-Newton subgradient method for ill-conditioned problems

TL;DR

This paper introduces a preconditioned subgradient method for optimizing composite functions that achieves linear convergence under certain conditions, with oracle complexity unaffected by reparameterizations.

Contribution

It presents a novel preconditioned subgradient method with linear convergence for composite optimization problems satisfying specific regularity conditions.

Findings

Method converges linearly when conditions are met.

Oracle complexity remains invariant under reparameterizations.

Applicable to problems with locally Lipschitz and semismooth functions.

Abstract

We analyze a preconditioned subgradient method for optimizing composite functions $h \circ c$, where $h$ is a locally Lipschitz function and $c$ is a smooth nonlinear mapping. We prove that when $c$ satisfies a constant rank property and $h$ is semismooth and sharp on the image of $c$, the method converges linearly. In contrast to standard subgradient methods, its oracle complexity is invariant under reparameterizations of $c$.