Solving the OSCAR and SLOPE Models Using a Semismooth Newton-Based Augmented Lagrangian Method

TL;DR

This paper introduces a semismooth Newton-based augmented Lagrangian method to efficiently solve OSCAR and SLOPE models, significantly improving speed and robustness in high-dimensional feature selection tasks.

Contribution

The paper develops a novel semismooth Newton-based algorithm that effectively handles the non-smooth, inseparable regularizers in OSCAR and SLOPE models, reducing computational complexity.

Findings

Algorithm outperforms existing methods in speed.

Algorithm demonstrates higher robustness.

Effective in high-dimensional data analysis.

Abstract

The octagonal shrinkage and clustering algorithm for regression (OSCAR), equipped with the $\ell_1$-norm and a pair-wise $\ell_{\infty}$-norm regularizer, is a useful tool for feature selection and grouping in high-dimensional data analysis. The computational challenge posed by OSCAR, for high dimensional and/or large sample size data, has not yet been well resolved due to the non-smoothness and inseparability of the regularizer involved. In this paper, we successfully resolve this numerical challenge by proposing a sparse semismooth Newton-based augmented Lagrangian method to solve the more general SLOPE (the sorted L-one penalized estimation) model. By appropriately exploiting the inherent sparse and low-rank property of the generalized Jacobian of the semismooth Newton system in the augmented Lagrangian subproblem, we show how the computational complexity can be substantially reduced. Our algorithm presents a notable advantage in the high-dimensional statistical regression settings. Numerical experiments are conducted on real data sets, and the results demonstrate that our algorithm is far superior, in both speed and robustness, than the existing state-of-the-art algorithms based on first-order iterative schemes, including the widely used accelerated proximal gradient (APG) method and the alternating direction method of multipliers (ADMM).