A First-Order Primal-Dual Method for Nonconvex Constrained Optimization Based On the Augmented Lagrangian

TL;DR

This paper introduces a novel first-order primal-dual algorithm, NAPP-AL, for nonconvex constrained optimization, demonstrating convergence to stationary points with sublinear and linear rates under certain conditions.

Contribution

The paper proposes the NAPP-AL method, a new flexible primal-dual approach for nonconvex nonsmooth problems, with proven convergence rates and conditions.

Findings

Converges to stationary solutions at o(1/√k) rate.

Achieves linear convergence under VP-EB condition.

Links Kurdyka-Lojasiewicz property to convergence conditions.

Abstract

Nonlinearly constrained nonconvex and nonsmooth optimization models play an increasingly important role in machine learning, statistics and data analytics. In this paper, based on the augmented Lagrangian function we introduce a flexible first-order primal-dual method, to be called nonconvex auxiliary problem principle of augmented Lagrangian (NAPP-AL), for solving a class of nonlinearly constrained nonconvex and nonsmooth optimization problems. We demonstrate that NAPP-AL converges to a stationary solution at the rate of o(1/\sqrt{k}), where k is the number of iterations. Moreover, under an additional error bound condition (to be called VP-EB in the paper), we further show that the convergence rate is in fact linear. Finally, we show that the famous Kurdyka- Lojasiewicz property and the metric subregularity imply the afore-mentioned VP-EB condition.