AL-CoLe: Augmented Lagrangian for Constrained Learning

TL;DR

This paper revisits Augmented Lagrangian methods for constrained learning, establishing strong duality, convergence, and generalization guarantees, and demonstrating effectiveness in fairness classification tasks.

Contribution

It provides new theoretical insights and practical algorithms for constrained learning using Augmented Lagrangian methods in non-convex settings.

Findings

Strong duality under mild conditions

Convergence of dual ascent algorithms

Effective fairness constrained classification results

Abstract

Despite the non-convexity of most modern machine learning parameterizations, Lagrangian duality has become a popular tool for addressing constrained learning problems. We revisit Augmented Lagrangian methods, which aim to mitigate the duality gap in non-convex settings while requiring only minimal modifications, and have remained comparably unexplored in constrained learning settings. We establish strong duality results under mild conditions, prove convergence of dual ascent algorithms to feasible and optimal primal solutions, and provide PAC-style generalization guarantees. Finally, we demonstrate its effectiveness on fairness constrained classification tasks.