A projection-free dynamics for nonsmooth composite optimization

TL;DR

This paper introduces a projection-free primal-dual dynamics for nonsmooth composite optimization, utilizing mirror descent and proximal augmented Lagrangian methods to achieve exponential convergence under general convex constraints.

Contribution

It develops a novel continuous-time primal-dual dynamics that avoids gradient projection, extending convergence guarantees to more general convex constraints with improved analysis.

Findings

Achieves exponential convergence for constrained nonsmooth optimization.

Extends convergence analysis to general convex equality-inequality constraints.

Provides a more efficient and analytically tractable alternative to gradient projection methods.

Abstract

This paper proposes a projection-free primal-dual dynamics for the nonsmooth composite optimization problems with equality and inequality constraints. To deal with optimization constraints, this paper departs from the use of gradient projection method, but resorts to the idea of mirror descent to design a continuous-time smooth optimization dynamics which advantageously leads to easier convergence analysis and more efficient numerical simulation. Also, the strategy of proximal augmented Lagrangian (PAL$^{\dag}$) is extended to incorporate general convex equality-inequality constraints and the strong convexity-concavity of the primal-dual variables is achieved, ensuring exponential convergence of the resulting algorithm. Furthermore, the convergence result in this paper extends existing exponential convergence which either takes no account of constraints or considers only affine linear constraints, and it also enhances existing asymptotic convergence under convex constraints which unfortunately depends on the complex gradient projection scheme.