Block Alternating Bregman Majorization Minimization with Extrapolation

TL;DR

This paper introduces a novel accelerated block alternating Bregman majorization-minimization method with extrapolation for nonsmooth nonconvex optimization, demonstrating convergence and effectiveness on matrix factorization tasks.

Contribution

It proposes a new Bregman-based accelerated algorithm with extrapolation for block nonconvex problems, extending existing methods and proving convergence.

Findings

Proves subsequential convergence to stationary points.

Establishes global convergence under stronger conditions.

Shows effectiveness on orthogonal nonnegative matrix factorization.

Abstract

In this paper, we consider a class of nonsmooth nonconvex optimization problems whose objective is the sum of a block relative smooth function and a proper and lower semicontinuous block separable function. Although the analysis of block proximal gradient (BPG) methods for the class of block $L$-smooth functions have been successfully extended to Bregman BPG methods that deal with the class of block relative smooth functions, accelerated Bregman BPG methods are scarce and challenging to design. Taking our inspiration from Nesterov-type acceleration and the majorization-minimization scheme, we propose a block alternating Bregman Majorization-Minimization framework with Extrapolation (BMME). We prove subsequential convergence of BMME to a first-order stationary point under mild assumptions, and study its global convergence under stronger conditions. We illustrate the effectiveness of BMME on the penalized orthogonal nonnegative matrix factorization problem.