Screening Cut Generation for Sparse Ridge Regression

TL;DR

This paper introduces a novel Screening Cut Generation method for sparse ridge regression that improves the elimination of non-optimal solutions, enhancing the efficiency of solving high-dimensional, complex instances.

Contribution

The paper proposes a new SCG method that uses convex relaxation to identify optimal feature combinations, outperforming existing safe screening approaches.

Findings

SCG effectively eliminates non-optimal solutions in sparse ridge regression.

Numerical experiments show SCG's superior performance on high-dimensional and challenging instances.

SCG enhances the pre-processing step in branch-and-bound algorithms for better solution efficiency.

Abstract

Sparse ridge regression is widely utilized in modern data analysis and machine learning. However, computing globally optimal solutions for sparse ridge regression is challenging, particularly when samples are arbitrarily given or generated under weak modeling assumptions. This paper proposes a novel cut-generation method, Screening Cut Generation (SCG), to eliminate non-optimal solutions for arbitrarily given samples. In contrast to recent safe variable screening approaches, SCG offers superior screening capability by identifying whether a specific $\{\pm 1\}$ combination of multiple features (binaries) lies in the set of optimal solutions. This identification is based on a convex relaxation solution rather than directly solving the original sparse ridge regression. Hence, the cuts generated by SCG can be applied in the pre-processing step of branch-and-bound and its variants to construct safe outer approximations of the optimal solution set. Numerical experiments are reported to validate the theoretical results and demonstrate the efficiency of SCG, particularly in hard real instances and synthetic instances with high dimensions, low ridge regularization parameters, or challenging modeling assumptions.