Accelerating Non-Negative and Bounded-Variable Linear Regression Algorithms with Safe Screening

TL;DR

This paper introduces a safe screening technique to accelerate algorithms for non-negative and bounded-variable linear regression by identifying saturated coordinates during iterations, leading to significant computational speedups.

Contribution

It presents a novel safe screening method with theoretical guarantees for non-negative and bounded-variable linear regression, improving solver efficiency.

Findings

Significant acceleration in solving regression problems

Theoretical guarantees ensure correctness of screening

Effective on both synthetic and real datasets

Abstract

Non-negative and bounded-variable linear regression problems arise in a variety of applications in machine learning and signal processing. In this paper, we propose a technique to accelerate existing solvers for these problems by identifying saturated coordinates in the course of iterations. This is akin to safe screening techniques previously proposed for sparsity-regularized regression problems. The proposed strategy is provably safe as it provides theoretical guarantees that the identified coordinates are indeed saturated in the optimal solution. Experimental results on synthetic and real data show compelling accelerations for both non-negative and bounded-variable problems.