Efficient Path Algorithms for Clustered Lasso and OSCAR

TL;DR

This paper introduces efficient path algorithms for clustered Lasso and OSCAR that significantly reduce computational costs by exploiting symmetry and graph theory, enabling better feature clustering in high-dimensional regression.

Contribution

It presents novel, more efficient algorithms for constructing solution paths in clustered Lasso and OSCAR, improving computational performance over existing methods.

Findings

Algorithms outperform existing methods in numerical experiments.

Computational costs are significantly reduced by exploiting symmetry.

Simple conditions for subgradient equations are derived using graph theory.

Abstract

In high dimensional regression, feature clustering by their effects on outcomes is often as important as feature selection. For that purpose, clustered Lasso and octagonal shrinkage and clustering algorithm for regression (OSCAR) are used to make feature groups automatically by pairwise $L_1$ norm and pairwise $L_\infty$ norm, respectively. This paper proposes efficient path algorithms for clustered Lasso and OSCAR to construct solution paths with respect to their regularization parameters. Despite too many terms in exhaustive pairwise regularization, their computational costs are reduced by using symmetry of those terms. Simple equivalent conditions to check subgradient equations in each feature group are derived by some graph theories. The proposed algorithms are shown to be more efficient than existing algorithms in numerical experiments.