A Global Two-stage Algorithm for Non-convex Penalized High-dimensional Linear Regression Problems

TL;DR

This paper introduces a global two-stage algorithm for high-dimensional linear regression with non-convex penalties, leveraging the DC property and primal dual active set methods to ensure convergence and computational efficiency.

Contribution

The paper develops a novel global two-stage algorithm for non-convex penalized regression, with proven convergence and improved computational performance over existing methods.

Findings

Algorithm converges globally to a d-stationary point.

Outperforms SSN and coordinate descent methods in simulations.

Proven theoretical convergence guarantees.

Abstract

By the asymptotic oracle property, non-convex penalties represented by minimax concave penalty (MCP) and smoothly clipped absolute deviation (SCAD) have attracted much attentions in high-dimensional data analysis, and have been widely used in signal processing, image restoration, matrix estimation, etc. However, in view of their non-convex and non-smooth characteristics, they are computationally challenging. Almost all existing algorithms converge locally, and the proper selection of initial values is crucial. Therefore, in actual operation, they often combine a warm-starting technique to meet the rigid requirement that the initial value must be sufficiently close to the optimal solution of the corresponding problem. In this paper, based on the DC (difference of convex functions) property of MCP and SCAD penalties, we aim to design a global two-stage algorithm for the high-dimensional least squares linear regression problems. A key idea for making the proposed algorithm to be efficient is to use the primal dual active set with continuation (PDASC) method, which is equivalent to the semi-smooth Newton (SSN) method, to solve the corresponding sub-problems. Theoretically, we not only prove the global convergence of the proposed algorithm, but also verify that the generated iterative sequence converges to a d-stationary point. In terms of computational performance, the abundant research of simulation and real data show that the algorithm in this paper is superior to the latest SSN method and the classic coordinate descent (CD) algorithm for solving non-convex penalized high-dimensional linear regression problems.