Scalable First-order Method for Certifying Optimal k-Sparse GLMs

TL;DR

This paper introduces a scalable first-order method using proximal gradient algorithms to efficiently compute dual bounds, enabling certification of optimal sparse GLMs at large scale.

Contribution

We develop a novel first-order proximal gradient approach with an exact, log-linear time proximal operator for certifying optimality in sparse GLMs, improving scalability.

Findings

Significantly accelerates dual bound computations.

Effective in certifying optimality for large-scale sparse GLMs.

Outperforms existing methods in speed and scalability.

Abstract

This paper investigates the problem of certifying optimality for sparse generalized linear models (GLMs), where sparsity is enforced through an $\ell_0$ cardinality constraint. While branch-and-bound (BnB) frameworks can certify optimality by pruning nodes using dual bounds, existing methods for computing these bounds are either computationally intensive or exhibit slow convergence, limiting their scalability to large-scale problems. To address this challenge, we propose a first-order proximal gradient algorithm designed to solve the perspective relaxation of the problem within a BnB framework. Specifically, we formulate the relaxed problem as a composite optimization problem and demonstrate that the proximal operator of the non-smooth component can be computed exactly in log-linear time complexity, eliminating the need to solve a computationally expensive second-order cone program. Furthermore, we introduce a simple restart strategy that enhances convergence speed while maintaining low per-iteration complexity. Extensive experiments on synthetic and real-world datasets show that our approach significantly accelerates dual bound computations and is highly effective in providing optimality certificates for large-scale problems.