GPU-friendly and Linearly Convergent First-order Methods for Certifying Optimal $k$-sparse GLMs

TL;DR

This paper introduces a GPU-accelerated, linearly convergent first-order method for certifying optimal sparse generalized linear models, significantly improving scalability and efficiency over existing approaches.

Contribution

It develops a unified proximal framework with a duality gap-based restart scheme, enabling fast, scalable certification of optimality for sparse GLMs, including specialized routines for the perspective regularizer.

Findings

Orders-of-magnitude faster dual-bound computations

Enhanced scalability of branch-and-bound on large instances

GPU acceleration of the proposed methods

Abstract

We investigate the problem of certifying optimality for sparse generalized linear models (GLMs), where sparsity is enforced through a cardinality constraint. While Branch-and-Bound (BnB) frameworks can certify optimality using perspective relaxations, existing methods for solving these relaxations are computationally intensive, limiting their scalability. To address this challenge, we reformulate the relaxations as composite optimization problems and develop a unified proximal framework that is both linearly convergent and computationally efficient. Under specific geometric regularity conditions, our analysis links primal quadratic growth to dual quadratic decay, yielding error bounds that make the Fenchel duality gap a sharp proxy for progress towards the solution set. This leads to a duality gap-based restart scheme that upgrades a broad class of sublinear proximal methods to provably linearly convergent methods, and applies beyond the sparse GLM setting. For the implicit perspective regularizer, we further derive specialized routines to evaluate the regularizer and its proximal operator exactly in log-linear time, avoiding costly generic conic solvers. The resulting iterations are dominated by matrix--vector multiplications, which enables GPU acceleration. Experiments on synthetic and real-world datasets show orders-of-magnitude faster dual-bound computations and substantially improved BnB scalability on large instances.