Four algorithms to solve symmetric multi-type non-negative matrix tri-factorization problem

TL;DR

This paper introduces four algorithms for solving the symmetric multi-type non-negative matrix tri-factorization problem, demonstrating their effectiveness on synthetic and real data, with ADAM often performing best given enough time.

Contribution

The paper develops and compares four novel algorithms for SNMTF, including implementations in MATLAB and Python, and provides extensive numerical evaluation on multiple datasets.

Findings

All four methods perform satisfactorily with sufficient time.

ADAM often yields the lowest MSE when given enough time.

FPM converges fastest initially, providing the best MSE under limited computation time.

Abstract

In this paper, we consider the symmetric multi-type non-negative matrix tri-factorization problem (SNMTF), which attempts to factorize several symmetric non-negative matrices simultaneously. This can be considered as a generalization of the classical non-negative matrix tri-factorization problem and includes a non-convex objective function which is a multivariate sixth degree polynomial and a has convex feasibility set. It has a special importance in data science, since it serves as a mathematical model for the fusion of different data sources in data clustering. We develop four methods to solve the SNMTF. They are based on four theoretical approaches known from the literature: the fixed point method (FPM), the block-coordinate descent with projected gradient (BCD), the gradient method with exact line search (GM-ELS) and the adaptive moment estimation method (ADAM). For each of these methods we offer a software implementation: for the former two methods we use Matlab and for the latter Python with the TensorFlow library. We test these methods on three data-sets: the synthetic data-set we generated, while the others represent real-life similarities between different objects. Extensive numerical results show that with sufficient computing time all four methods perform satisfactorily and ADAM most often yields the best mean square error ($\mathrm{MSE}$). However, if the computation time is limited, FPM gives the best $\mathrm{MSE}$ because it shows the fastest convergence at the beginning. All data-sets and codes are publicly available on our GitLab profile.