A Cyclic Coordinate Descent Algorithm for lq Regularization

TL;DR

This paper introduces a cyclic coordinate descent algorithm for $l_q$ regularization ($0<q<1$), demonstrating its convergence properties and efficiency in sparse modeling tasks.

Contribution

The paper proposes a novel CCD algorithm for $l_q$ regularization with proven convergence to stationary points and local minima, enhancing sparse modeling techniques.

Findings

The CCD algorithm converges globally to a stationary point.

Under certain conditions, it converges to a local minimizer.

Numerical experiments show the algorithm's efficiency.

Abstract

In recent studies on sparse modeling, $l_q$ ($0<q<1$) regularization has received considerable attention due to its superiorities on sparsity-inducing and bias reduction over the $l_1$ regularization.In this paper, we propose a cyclic coordinate descent (CCD) algorithm for $l_q$ regularization. Our main result states that the CCD algorithm converges globally to a stationary point as long as the stepsize is less than a positive constant. Furthermore, we demonstrate that the CCD algorithm converges to a local minimizer under certain additional conditions. Our numerical experiments demonstrate the efficiency of the CCD algorithm.