An Extrapolated Iteratively Reweighted l1 Method with Complexity Analysis

TL;DR

This paper introduces a new extrapolated iteratively reweighted l1 algorithm with proven convergence and local linear complexity, applicable to non-Lipschitz regularizers, and demonstrates its efficiency through numerical tests.

Contribution

It proposes a novel reweighted l1 algorithm with extrapolation that does not require Lipschitz differentiability or bounded smoothing parameters, with proven convergence and complexity analysis.

Findings

Converges uniquely to a stationary point.

Achieves local linear complexity.

Demonstrates efficiency in numerical experiments.

Abstract

The iteratively reweighted l1 algorithm is a widely used method for solving various regularization problems, which generally minimize a differentiable loss function combined with a nonconvex regularizer to induce sparsity in the solution. However, the convergence and the complexity of iteratively reweighted l1 algorithms is generally difficult to analyze, especially for non-Lipschitz differentiable regularizers such as nonconvex lp norm regularization. In this paper, we propose, analyze and test a reweighted l1 algorithm combined with the extrapolation technique under the assumption of Kurdyka-Lojasiewicz (KL) property on the objective. Unlike existing iteratively reweighted l1 algorithms with extrapolation, our method does not require the Lipschitz differentiability on the regularizers nor the smoothing parameters in the weights bounded away from zero. We show the proposed algorithm converges uniquely to a stationary point of the regularization problem and has local linear complexity--a much stronger result than existing ones. Our numerical experiments show the efficiency of our proposed method.