Leveraging Two Reference Functions in Block Bregman Proximal Gradient Descent for Non-convex and Non-Lipschitz Problems

TL;DR

This paper introduces a novel block-wise two-reference Bregman proximal gradient method for non-convex, non-Lipschitz problems, achieving faster convergence and improved performance in applications like nonnegative matrix factorization.

Contribution

The paper proposes a new B2B method using two reference functions, providing closed-form solutions and establishing global convergence rates for non-convex, non-Lipschitz problems.

Findings

The B2B method converges globally under various block selection rules.

The convergence rate is faster than cyclic variants, specifically $O(rac{ ext{sqrt}(s)}{ ext{sqrt}(k)})$.

Numerical results demonstrate the method's superiority in NMF problems.

Abstract

In the applications of signal processing and data analytics, there is a wide class of non-convex problems whose objective function is freed from the common global Lipschitz continuous gradient assumption (e.g., the nonnegative matrix factorization (NMF) problem). Recently, this type of problem with some certain special structures has been solved by Bregman proximal gradient (BPG). This inspires us to propose a new Block-wise two-references Bregman proximal gradient (B2B) method, which adopts two reference functions so that a closed-form solution in the Bregman projection is obtained. Based on the relative smoothness, we prove the global convergence of the proposed algorithms for various block selection rules. In particular, we establish the global convergence rate of $O(\frac{\sqrt{s}}{\sqrt{k}})$ for the greedy and randomized block updating rule for B2B, which is $O(\sqrt{s})$ times faster than the cyclic variant, i.e., $O(\frac{s}{\sqrt{k}} )$, where $s$ is the number of blocks, and $k$ is the number of iterations. Multiple numerical results are provided to illustrate the superiority of the proposed B2B compared to the state-of-the-art works in solving NMF problems.