Fast and Effective Computation of Generalized Symmetric Matrix Factorization

TL;DR

This paper introduces a novel nonmonotone alternating method for efficiently solving a broad class of generalized symmetric matrix factorization problems, with proven convergence and demonstrated practical performance.

Contribution

It develops an exact penalty and relaxation framework for symmetric matrix factorization, and proposes a new algorithm with convergence guarantees and improved efficiency.

Findings

Algorithm converges globally to a stationary point

Numerical experiments show high efficiency on real datasets

The method outperforms existing approaches in speed and accuracy

Abstract

In this paper, we study a nonconvex, nonsmooth, and non-Lipschitz generalized symmetric matrix factorization model that unifies a broad class of matrix factorization formulations arising in machine learning, image science, engineering, and related areas. We first establish two exactness properties. On the modeling side, we prove an exact penalty property showing that, under suitable conditions, the symmetry-inducing quadratic penalty enforces symmetry whenever the penalty parameter is sufficiently large but finite, thereby exactly recovering the associated symmetric formulation. On the algorithmic side, we introduce an auxiliary-variable splitting formulation and establish an exact relaxation relationship that rigorously links stationary points of the original objective function to those of a relaxed potential function. Building on these exactness properties, we propose an average-type nonmonotone alternating updating method (A-NAUM) based on the relaxed potential function. At each iteration, A-NAUM alternately updates the two factor blocks by (approximately) minimizing the potential function, while the auxiliary block is updated in closed form. To ensure the convergence and enhance practical performance, we further incorporate an average-type nonmonotone line search and show that it is well-defined under mild conditions. Moreover, based on the Kurdyka-{\L}ojasiewicz property and its associated exponent, we establish global convergence of the entire sequence to a stationary point and derive convergence rate results. Finally, numerical experiments on real datasets demonstrate the efficiency of A-NAUM.