Active set identification and rapid convergence for degenerate primal-dual problems

TL;DR

This paper provides a theoretical framework for active set identification in primal-dual convex optimization methods, showing nonasymptotic guarantees without strict complementarity assumptions.

Contribution

It introduces mild geometric conditions ensuring active set identification and rapid convergence, applicable to various popular algorithms without relying on nondegeneracy.

Findings

Nonasymptotic guarantees for active set identification

Applicable to multiple primal-dual algorithms

Does not require strict complementarity

Abstract

Primal-dual methods for solving convex optimization problems with functional constraints often exhibit a distinct two-stage behavior. Initially, they converge towards a solution at a sublinear rate. Then, after a certain point, the method identifies the set of active constraints and the convergence enters a faster local linear regime. Theory characterizing this phenomenon spans over three decades. However, most existing work only guarantees eventual identification of the active set and relies heavily on nondegeneracy conditions, such as strict complementarity, which often fail to hold in practice. We characterize mild conditions on the problem geometry and the algorithm under which this phenomenon provably occurs. Our guarantees are entirely nonasymptotic and, importantly, do not rely on strict complementarity. Our framework encompasses several widely-used algorithms, including the proximal point method, the primal-dual hybrid gradient method, the alternating direction method of multipliers, and the extragradient method.