The Smoothing Proximal Gradient Algorithm with Extrapolation for the Relaxation of $l_0$ Regularization Problem

TL;DR

This paper introduces a smoothing proximal gradient algorithm with extrapolation (SPGE) for the relaxed $l_0$ regularization problem, proving convergence to stationary points and demonstrating faster numerical performance over previous methods.

Contribution

The paper develops a novel SPGE algorithm with proven convergence properties for the $l_0$ relaxation model, improving efficiency over existing algorithms.

Findings

SPGE converges to a lifted stationary point.

The entire sequence generated by SPGE converges.

Numerical experiments show SPGE is faster than previous methods.

Abstract

In this paper, we consider the exact continuous relaxation model of $l_0$ regularization problem which was given by Bian and Chen (SIAM J. Numer. Anal 58(1): 858-883, 2020) and propose a smoothing proximal gradient algorithm with extrapolation (SPGE) for this kind of problem. We show that any accumulation point of the sequence generated by SPGE algorithm is a lifted stationary point of the relaxation model for a fairly general choice of extrapolation parameter. Moreover, it is shown that the whole sequence generated by SPGE algorithm converges to a lifted stationary point of the relaxation model. The convergence rate of $o(\frac{1}{k^{1-\sigma}})(\sigma\in(0,1))$ with respect to squared proximal residual is established. Finally, we conduct three important numerical experiments to illustrate the faster efficiency of the SPGE algorithm compared with the smoothing proximal gradient(SPG) algorithm proposed by Bian and Chen (SIAM J. Numer. Anal 58(1):858-883, 2020).