A Non-monotone Alternating Updating Method for A Class of Matrix Factorization Problems

TL;DR

This paper introduces a non-monotone alternating updating algorithm for a broad class of matrix factorization problems, demonstrating improved efficiency and convergence properties in non-negative matrix factorization and matrix completion tasks.

Contribution

The paper proposes a novel non-monotone alternating updating method based on a potential function, with theoretical convergence guarantees and practical efficiency improvements.

Findings

Method outperforms existing algorithms in numerical experiments.

Converges to stationary points under mild conditions.

Effective for non-negative matrix factorization and matrix completion.

Abstract

In this paper we consider a general matrix factorization model which covers a large class of existing models with many applications in areas such as machine learning and imaging sciences. To solve this possibly nonconvex, nonsmooth and non-Lipschitz problem, we develop a non-monotone alternating updating method based on a potential function. Our method essentially updates two blocks of variables in turn by inexactly minimizing this potential function, and updates another auxiliary block of variables using an explicit formula. The special structure of our potential function allows us to take advantage of efficient computational strategies for non-negative matrix factorization to perform the alternating minimization over the two blocks of variables. A suitable line search criterion is also incorporated to improve the numerical performance. Under some mild conditions, we show that the line search criterion is well defined, and establish that the sequence generated is bounded and any cluster point of the sequence is a stationary point. Finally, we conduct some numerical experiments using real datasets to compare our method with some existing efficient methods for non-negative matrix factorization and matrix completion. The numerical results show that our method can outperform these methods for these specific applications.