Proximal Iterative Hard Thresholding Algorithm for Sparse Group $\ell_0$-Regularized Optimization with Box Constraint

TL;DR

This paper introduces a proximal iterative hard thresholding algorithm for solving non-convex sparse optimization problems with box constraints, demonstrating its convergence and efficiency through theoretical analysis and experiments.

Contribution

It proposes a novel proximal iterative hard thresholding algorithm for $ ext{l}_0$-regularized optimization with box constraints, including convergence analysis and experimental validation.

Findings

The algorithm converges globally.

It effectively handles group and element-wise sparsity.

Experiments show high efficiency and accuracy.

Abstract

This paper investigates a general class of problems in which a lower bounded smooth convex function incorporating $\ell_{0}$ and $\ell_{2,0}$ regularization is minimized over a box constraint. Although such problems arise frequently in practical applications, their inherent non-convexity poses significant challenges for solution methods. In particular, we focus on the proximal operator associated with these regularizations, which incorporates both group-sparsity and element-wise sparsity terms. Besides, we introduce the concepts of $\tau$-stationary point and support optimal (SO) point then analyze their relationship with the minimizer of the considered problem. Based on the proximal operator, we propose a novel proximal iterative hard thresholding algorithm to solve the problem. Furthermore, we establish the global convergence and the computational complexity analysis of the proposed method. Finally, extensive experiments demonstrate the effectiveness and efficiency of our method.