Accelerating Nonnegative Matrix Factorization Algorithms using Extrapolation

TL;DR

This paper introduces a novel extrapolation-based framework to significantly speed up nonnegative matrix factorization algorithms, demonstrating improved performance on synthetic, image, and document datasets.

Contribution

The paper presents a new extrapolation scheme applied to existing NMF algorithms, enhancing their convergence speed in non-convex optimization contexts.

Findings

Accelerated convergence of NMF algorithms demonstrated

Effective on synthetic, image, and document datasets

Novel use of extrapolation in two-block coordinate descent methods

Abstract

In this paper, we propose a general framework to accelerate significantly the algorithms for nonnegative matrix factorization (NMF). This framework is inspired from the extrapolation scheme used to accelerate gradient methods in convex optimization and from the method of parallel tangents. However, the use of extrapolation in the context of the two-block exact coordinate descent algorithms tackling the non-convex NMF problems is novel. We illustrate the performance of this approach on two state-of-the-art NMF algorithms, namely, accelerated hierarchical alternating least squares (A-HALS) and alternating nonnegative least squares (ANLS), using synthetic, image and document data sets.