A Gauss-Seidel Iterative Thresholding Algorithm for lq Regularized Least Squares Regression

TL;DR

This paper introduces GAITA, a Gauss-Seidel based iterative thresholding algorithm for solving non-convex l_q regularized least squares regression, demonstrating faster convergence and convergence properties under weaker conditions.

Contribution

The paper proposes a novel Gauss-Seidel iterative thresholding algorithm (GAITA) for l_qLS, improving convergence speed and theoretical guarantees over classical methods.

Findings

GAITA converges faster than classical algorithms.

Support set and sign converge within finite iterations.

GAITA converges to a local minimizer under certain conditions.

Abstract

In recent studies on sparse modeling, $l_q$ ($0<q<1$) regularized least squares regression ($l_q$LS) has received considerable attention due to its superiorities on sparsity-inducing and bias-reduction over the convex counterparts. In this paper, we propose a Gauss-Seidel iterative thresholding algorithm (called GAITA) for solution to this problem. Different from the classical iterative thresholding algorithms using the Jacobi updating rule, GAITA takes advantage of the Gauss-Seidel rule to update the coordinate coefficients. Under a mild condition, we can justify that the support set and sign of an arbitrary sequence generated by GAITA will converge within finite iterations. This convergence property together with the Kurdyka-{\L}ojasiewicz property of ($l_q$LS) naturally yields the strong convergence of GAITA under the same condition as above, which is generally weaker than the condition for the convergence of the classical iterative thresholding algorithms. Furthermore, we demonstrate that GAITA converges to a local minimizer under certain additional conditions. A set of numerical experiments are provided to show the effectiveness, particularly, much faster convergence of GAITA as compared with the classical iterative thresholding algorithms.