Input Guided Multiple Deconstruction Single Reconstruction neural network models for Matrix Factorization

TL;DR

This paper introduces two neural network models inspired by NMF for effective low-rank data approximation, emphasizing non-negativity constraints and local structure preservation, validated through extensive experiments and complexity analysis.

Contribution

The paper proposes novel neural network models for NMF and relaxed NMF that incorporate input guidance and deconstruction-reconstruction mechanisms, enhancing low-dimensional embedding quality.

Findings

Models outperform nine existing algorithms on five datasets.

Both models effectively preserve local data structure.

The models demonstrate favorable computational complexity and convergence.

Abstract

Referring back to the original text in the course of hierarchical learning is a common human trait that ensures the right direction of learning. The models developed based on the concept of Non-negative Matrix Factorization (NMF), in this paper are inspired by this idea. They aim to deal with high-dimensional data by discovering its low rank approximation by determining a unique pair of factor matrices. The model, named Input Guided Multiple Deconstruction Single Reconstruction neural network for Non-negative Matrix Factorization (IG-MDSR-NMF), ensures the non-negativity constraints of both factors. Whereas Input Guided Multiple Deconstruction Single Reconstruction neural network for Relaxed Non-negative Matrix Factorization (IG-MDSR-RNMF) introduces a novel idea of factorization with only the basis matrix adhering to the non-negativity criteria. This relaxed version helps the model to learn more enriched low dimensional embedding of the original data matrix. The competency of preserving the local structure of data in its low rank embedding produced by both the models has been appropriately verified. The superiority of low dimensional embedding over that of the original data justifying the need for dimension reduction has been established. The primacy of both the models has also been validated by comparing their performances separately with that of nine other established dimension reduction algorithms on five popular datasets. Moreover, computational complexity of the models and convergence analysis have also been presented testifying to the supremacy of the models.