Second-order orthant-based methods with enriched Hessian information for sparse $\ell_1$-optimization

TL;DR

This paper introduces a second-order orthantwise method with enriched Hessian information for sparse -optimization, improving active set identification and efficiency over existing algorithms.

Contribution

It proposes a novel second-order algorithm using partial Huber regularization for better active set detection in -regularized problems.

Findings

Faster active set identification compared to existing methods

Reduced algorithm is equivalent to a semismooth Newton method

Demonstrated efficiency through computational experiments

Abstract

We present a second order algorithm, based on orthantwise directions, for solving optimization problems involving the sparsity enhancing $\ell_1$-norm. The main idea of our method consists in modifying the descent orthantwise directions by using second order information both of the regular term and (in weak sense) of the $\ell_1$-norm. The weak second order information behind the $\ell_1$-term is incorporated via a partial Huber regularization. One of the main features of our algorithm consists in a faster identification of the active set. We also prove that a reduced version of our method is equivalent to a semismooth Newton algorithm applied to the optimality condition, under a specific choice of the algorithm parameters. We present several computational experiments to show the efficiency of our approach compared to other state-of-the-art algorithms.