Convergence of dual ascent in non-convex/non-differentiable optimization

TL;DR

This paper extends the convergence analysis of dual ascent algorithms to non-convex and non-differentiable functionals, introduces an augmented version with better convergence, and applies these methods to matrix low-rank approximation and frequency estimation.

Contribution

It provides new convergence results for dual ascent in non-convex/non-differentiable settings and introduces an augmented algorithm with improved properties.

Findings

Convergence results apply to non-convex and non-differentiable functionals.

The augmented algorithm shows superior convergence properties.

Numerical tests demonstrate effectiveness in frequency estimation.

Abstract

We revisit the classical dual ascent algorithm for minimization of convex functionals in the presence of linear constraints, and give convergence results which apply even for non-convex functionals. We describe limit points in terms of the convex envelope. We also introduce a new augmented version, which is shown to have superior convergence properties, and provide new results even for convex but non-differentiable objective functionals (as well as non-convex). The results are applied to low rank approximation of a given matrix, subject to linear constraints. In particular, letting the linear constraints enforce Hankel structure of the respective matrices, the algorithms can be applied to complex frequency estimation. We provide numerical tests in this setting.