A GPU-accelerated Nonlinear Branch-and-Bound Framework for Sparse Linear Models

TL;DR

This paper introduces a GPU-accelerated branch-and-bound framework for solving exact sparse linear regression problems with an - penalty, significantly improving runtime efficiency on high-dimensional datasets.

Contribution

The paper presents a novel GPU-optimized branch-and-bound algorithm for sparse linear regression, utilizing interval relaxation and batched matrix operations for enhanced parallelism.

Findings

Achieves faster runtimes than CPU-based methods and existing solvers.

Demonstrates effectiveness on synthetic and real datasets.

Provides practical GPU implementation strategies for sparse regression.

Abstract

We study exact sparse linear regression with an $\ell_0-\ell_2$ penalty and develop a branch-and-bound (BnB) algorithm explicitly designed for GPU execution. Starting from a perspective reformulation, we derive an interval relaxation that can be solved by ADMM with closed-form, coordinate-wise updates. We structure these updates so that the main work at each BnB node reduces to batched matrix-vector operations with a shared data matrix, enabling fine-grained parallelism across coordinates and coarse-grained parallelism across many BnB nodes on a single GPU. Feasible solutions (upper bounds) are generated by a projected gradient method on the active support, implemented in a batched fashion so that many candidate supports are updated in parallel on the GPU. We discuss practical design choices such as memory layout, batching strategies, and load balancing across nodes that are crucial for obtaining good utilization on modern GPUs. On synthetic and real high-dimensional datasets, our GPU-based approach achieves clear runtime improvements over a CPU implementation of our method, an existing specialized BnB method, and commercial MIP solvers.