Separable Bregman Framework for Sparsity Constrained Nonlinear Optimization

TL;DR

This paper introduces a new framework for sparsity-constrained nonlinear optimization using separable Bregman functions, leading to improved algorithms with theoretical guarantees, especially for compressed sensing applications.

Contribution

It develops a novel Bregman-based framework for sparsity constraints, extending hard-thresholding algorithms with new descent lemmas and optimality conditions.

Findings

New descent lemmas for smooth and relatively smooth functions

Enhanced algorithms with theoretical convergence guarantees

Successful application to compressed sensing problems

Abstract

This paper considers the minimization of a continuously differentiable function over a cardinality constraint. We focus on smooth and relatively smooth functions. These smoothness criteria result in new descent lemmas. Based on the new descent lemmas, novel optimality conditions and algorithms are developed, which extend the previously proposed hard-thresholding algorithms. We give a theoretical analysis of these algorithms and extend previous results on properties of iterative hard thresholding-like algorithms. In particular, we focus on the weighted $\ell_2$ norm, which requires efficient solution of convex subproblems. We apply our algorithms to compressed sensing problems to demonstrate the theoretical findings and the enhancements achieved through the proposed framework.