Quadratic Matrix Factorization with Applications to Manifold Learning

TL;DR

This paper introduces quadratic matrix factorization (QMF), a novel approach for modeling curved manifolds in data, demonstrating its effectiveness over existing local linear methods through theoretical analysis and experiments on real datasets.

Contribution

The paper proposes a new quadratic matrix factorization framework for manifold learning, including an optimization algorithm, regularization techniques, and application strategies.

Findings

QMF outperforms local linear methods on synthetic and real datasets.

The proposed algorithm has proven convergence properties.

Regularization improves model generalization.

Abstract

Matrix factorization is a popular framework for modeling low-rank data matrices. Motivated by manifold learning problems, this paper proposes a quadratic matrix factorization (QMF) framework to learn the curved manifold on which the dataset lies. Unlike local linear methods such as the local principal component analysis, QMF can better exploit the curved structure of the underlying manifold. Algorithmically, we propose an alternating minimization algorithm to optimize QMF and establish its theoretical convergence properties. Moreover, to avoid possible over-fitting, we then propose a regularized QMF algorithm and discuss how to tune its regularization parameter. Finally, we elaborate how to apply the regularized QMF to manifold learning problems. Experiments on a synthetic manifold learning dataset and two real datasets, including the MNIST handwritten dataset and a cryogenic electron microscopy dataset, demonstrate the superiority of the proposed method over its competitors.