An Exact Solution Path Algorithm for SLOPE and Quasi-Spherical OSCAR

TL;DR

This paper introduces an exact solution path algorithm for SLOPE, enabling precise tuning of regularization weights, and proposes QS-OSCAR, a new regularization sequence that improves feature clustering efficiency in high-dimensional regression.

Contribution

It develops a novel exact solution path algorithm for SLOPE and introduces QS-OSCAR, a new regularization sequence with improved clustering performance.

Findings

The solution path algorithm provides exact solutions for SLOPE.

QS-OSCAR outperforms other designs in feature clustering efficiency.

Numerical experiments confirm the effectiveness of QS-OSCAR.

Abstract

Sorted $L_1$ penalization estimator (SLOPE) is a regularization technique for sorted absolute coefficients in high-dimensional regression. By arbitrarily setting its regularization weights $\lambda$ under the monotonicity constraint, SLOPE can have various feature selection and clustering properties. On weight tuning, the selected features and their clusters are very sensitive to the tuning parameters. Moreover, the exhaustive tracking of their changes is difficult using grid search methods. This study presents a solution path algorithm that provides the complete and exact path of solutions for SLOPE in fine-tuning regularization weights. A simple optimality condition for SLOPE is derived and used to specify the next splitting point of the solution path. This study also proposes a new design of a regularization sequence $\lambda$ for feature clustering, which is called the quasi-spherical and octagonal shrinkage and clustering algorithm for regression (QS-OSCAR). QS-OSCAR is designed with a contour surface of the regularization terms most similar to a sphere. Among several regularization sequence designs, sparsity and clustering performance are compared through simulation studies. The numerical observations show that QS-OSCAR performs feature clustering more efficiently than other designs.