Global Convergence of Inexact Augmented Lagrangian Method for Zero-One Composite Optimization

TL;DR

This paper introduces an inexact augmented Lagrangian method (IALM) with convergence guarantees for solving the challenging zero-one composite optimization problem, which includes models like SVM and multi-label classification.

Contribution

It develops a globally convergent IALM for 0/1-COP, utilizing a novel proximal stationarity concept and a low-complexity subproblem solver, extending applicability to SVM, MLC, and MRC.

Findings

Proves convergence of IALM to local minimizers under certain conditions.

Establishes global convergence for SVM, MLC, and MRC when data matrix is full row rank.

Introduces a low-complexity zero-one Bregman alternating linearized minimization algorithm.

Abstract

We consider the problem of minimizing the sum of a smooth function and a composition of a zero-one loss function with a linear operator, namely zero-one composite optimization problem (0/1-COP). It is a versatile model including the support vector machine (SVM), multi-label classification (MLC), maximum rank correlation (MRC) and so on. However, due to the nonconvexity, discontinuity and NP-hardness of the 0/1-COP, it is intractable to design a globally convergent algorithm and the work attempting to solve it directly is scarce. In this paper, we first define and characterize the proximal stationarity to derive the minimum and the strongly exact penalization of the Lyapunov function, which is a variant of the augmented Lagrangian function for the 0/1-COP. Based on this, we propose an inexact augmented Lagrangian method (IALM) for solving 0/1-COP, where the subproblem is solved by the zero-one Bregman alternating linearized minimization (0/1-BALM) algorithm with low computational complexity. Under some suitable assumptions, we prove that the whole sequence generated by the IALM converges to the local minimizer of 0/1-COP. As a direct application, we obtain the global convergence of IALM under the assumption that the data matrix is full row rank for solving the SVM, MLC and MRC.