Coordinate Descent for MCP/SCAD Penalized Least Squares Converges Linearly

TL;DR

This paper proves that coordinate descent algorithms for MCP and SCAD penalized least squares problems converge at a linear rate, enhancing understanding of their efficiency in sparse signal recovery.

Contribution

It establishes the first linear convergence proof for coordinate descent methods applied to MCP and SCAD penalized least squares.

Findings

Coordinate descent converges linearly for MCP/SCAD penalized problems.

Theoretical guarantees improve understanding of algorithm efficiency.

Supports sparse signal recovery with nonconvex penalties.

Abstract

Recovering sparse signals from observed data is an important topic in signal/imaging processing, statistics and machine learning. Nonconvex penalized least squares have been attracted a lot of attentions since they enjoy nice statistical properties. Computationally, coordinate descent (CD) is a workhorse for minimizing the nonconvex penalized least squares criterion due to its simplicity and scalability. In this work, we prove the linear convergence rate to CD for solving MCP/SCAD penalized least squares problems.