A Second-Order Method for Strongly Convex L1-Regularization Problems

TL;DR

This paper introduces a primal-dual Newton Conjugate Gradients method for efficiently solving strongly convex L1-regularized problems, demonstrating favorable convergence and practical performance on synthetic and real-world datasets.

Contribution

It develops a robust second-order primal-dual method specifically designed for strongly convex L1-regularized problems, emphasizing efficiency and broad applicability.

Findings

Convergence properties of pdNCG are established.

Method performs well on synthetic sparse least-squares problems.

Effective on real-world machine learning tasks.

Abstract

In this paper a robust second-order method is developed for the solution of strongly convex l1-regularized problems. The main aim is to make the proposed method as inexpensive as possible, while even difficult problems can be efficiently solved. The proposed approach is a primal-dual Newton Conjugate Gradients (pdNCG) method. Convergence properties of pdNCG are studied and worst-case iteration complexity is established. Numerical results are presented on synthetic sparse least-squares problems and real world machine learning problems.