Learnable Similarity and Dissimilarity Guided Symmetric Non-Negative Matrix Factorization

TL;DR

This paper introduces a learnable similarity and dissimilarity guided symmetric non-negative matrix factorization method that improves clustering accuracy by adaptively learning the similarity matrix with reduced search space and enhanced discriminative power.

Contribution

It proposes a novel learnable similarity and dissimilarity matrix construction with a dual structure and orthogonality regularization, reducing complexity and improving clustering performance.

Findings

Outperforms nine state-of-the-art clustering methods on eight datasets.

Reduces similarity matrix learning dimension from O(n^2) to n-1.

Provides a theoretically guaranteed convergence of the optimization algorithm.

Abstract

Symmetric nonnegative matrix factorization (SymNMF) is a powerful tool for clustering, which typically uses the $k$-nearest neighbor ($k$-NN) method to construct similarity matrix. However, $k$-NN may mislead clustering since the neighbors may belong to different clusters, and its reliability generally decreases as $k$ grows. In this paper, we construct the similarity matrix as a weighted $k$-NN graph with learnable weight that reflects the reliability of each $k$-th NN. This approach reduces the search space of the similarity matrix learning to $n - 1$ dimension, as opposed to the $\mathcal{O}(n^2)$ dimension of existing methods, where $n$ represents the number of samples. Moreover, to obtain a discriminative similarity matrix, we introduce a dissimilarity matrix with a dual structure of the similarity matrix, and propose a new form of orthogonality regularization with discussions on its geometric interpretation and numerical stability. An efficient alternative optimization algorithm is designed to solve the proposed model, with theoretically guarantee that the variables converge to a stationary point that satisfies the KKT conditions. The advantage of the proposed model is demonstrated by the comparison with nine state-of-the-art clustering methods on eight datasets. The code is available at \url{https://github.com/lwl-learning/LSDGSymNMF}.