A Highly Efficient Algorithm for Solving Exclusive Lasso Problems

TL;DR

This paper introduces a highly efficient dual Newton method-based algorithm for solving large-scale exclusive lasso problems, significantly improving computational efficiency in high-dimensional feature selection tasks.

Contribution

The paper develops a novel dual Newton method with proximal point approach tailored for exclusive lasso, including systematic analysis of the proximal mapping and generalized Jacobian.

Findings

PPDNA outperforms existing algorithms in speed and accuracy

Extensive experiments validate the efficiency of the proposed method

The approach enhances practical applicability of exclusive lasso models

Abstract

The exclusive lasso (also known as elitist lasso) regularizer has become popular recently due to its superior performance on intra-group feature selection. Its complex nature poses difficulties for the computation of high-dimensional machine learning models involving such a regularizer. In this paper, we propose a highly efficient dual Newton method based proximal point algorithm (PPDNA) for solving large-scale exclusive lasso models. As important ingredients, we systematically study the proximal mapping of the weighted exclusive lasso regularizer and the corresponding generalized Jacobian. These results also make popular first-order algorithms for solving exclusive lasso models more practical. Extensive numerical results are presented to demonstrate the superior performance of the PPDNA against other popular numerical algorithms for solving the exclusive lasso problems.