Extrapolated Hard Thresholding Algorithms with Finite Length for Composite $\ell_0$ Penalized Problems

TL;DR

This paper introduces an extrapolated hard thresholding algorithm for sparse optimization with a specific penalty, proving finite-length iterates and convergence to local minimizers without relying on the Kurdyka-Łojasiewicz inequality, supported by numerical experiments.

Contribution

It proposes a novel extrapolated hard thresholding algorithm for composite penalized problems, establishing finite-length iterates and convergence properties without standard inequality assumptions.

Findings

Algorithm generates finite-length iterates with penalty.

Converges to an \u03b5-local minimizer for case.

Numerical experiments confirm theoretical convergence results.

Abstract

For a class of sparse optimization problems with the penalty function of $\|(\cdot)_+\|_0$, we first characterize its local minimizers and then propose an extrapolated hard thresholding algorithm to solve such problems. We show that the iterates generated by the proposed algorithm with $\epsilon>0$ (where $\epsilon$ is the dry friction coefficient) have finite length, without relying on the Kurdyka-{\L}ojasiewicz inequality. Furthermore, we demonstrate that the algorithm converges to an $\epsilon$-local minimizer of this problem. For the special case that $\epsilon=0$, we establish that any accumulation point of the iterates is a local minimizer of the problem. Additionally, we analyze the convergence when an error term is present in the algorithm, showing that the algorithm still converges in the same manner as before, provided that the errors asymptotically approach zero. Finally, we conduct numerical experiments to verify the theoretical results of the proposed algorithm.