On Quadratic Convergence of DC Proximal Newton Algorithm for Nonconvex Sparse Learning in High Dimensions

TL;DR

This paper introduces a DC proximal Newton algorithm for nonconvex sparse learning in high dimensions, achieving quadratic convergence and optimal statistical properties through multi-stage convex relaxation.

Contribution

The paper presents a novel DC proximal Newton method that combines convex relaxation with local convergence guarantees for high-dimensional sparse learning.

Findings

Achieves local quadratic convergence within each convex relaxation stage.

Obtains sparse approximate local optima with optimal statistical properties.

Supported by numerical experiments validating theoretical results.

Abstract

We propose a DC proximal Newton algorithm for solving nonconvex regularized sparse learning problems in high dimensions. Our proposed algorithm integrates the proximal Newton algorithm with multi-stage convex relaxation based on the difference of convex (DC) programming, and enjoys both strong computational and statistical guarantees. Specifically, by leveraging a sophisticated characterization of sparse modeling structures/assumptions (i.e., local restricted strong convexity and Hessian smoothness), we prove that within each stage of convex relaxation, our proposed algorithm achieves (local) quadratic convergence, and eventually obtains a sparse approximate local optimum with optimal statistical properties after only a few convex relaxations. Numerical experiments are provided to support our theory.