Rank Suggestion in Non-negative Matrix Factorization: Residual Sensitivity to Initial Conditions (RSIC)

TL;DR

This paper introduces RSIC, a new method for determining the rank in Non-negative Matrix Factorization by analyzing residual sensitivity to initial conditions, which improves robustness and applicability across diverse datasets.

Contribution

RSIC is a novel approach that uses residual sensitivity analysis to suggest multiple meaningful ranks in NMF, avoiding reliance on domain-specific assumptions.

Findings

RSIC effectively identifies relevant ranks aligned with data structure.

RSIC outperforms existing methods in accuracy and scalability.

RSIC is applicable to various data types like gene expression, images, and text.

Abstract

Determining the appropriate rank in Non-negative Matrix Factorization (NMF) is a critical challenge that often requires extensive parameter tuning and domain-specific knowledge. Traditional methods for rank determination focus on identifying a single optimal rank, which may not capture the complex structure inherent in real-world datasets. In this study, we introduce a novel approach called Residual Sensitivity to Intial Conditions (RSIC) that suggests potentially multiple ranks of interest by analyzing the sensitivity of the relative residuals (e.g. relative reconstruction error) to different initializations. By computing the Mean Coordinatewise Interquartile Range (MCI) of the residuals across multiple random initializations, our method identifies regions where the NMF solutions are less sensitive to initial conditions and potentially more meaningful. We evaluate RSIC on a diverse set of datasets, including single-cell gene expression data, image data, and text data, and compare it against current state-of-the-art existing rank determination methods. Our experiments demonstrate that RSIC effectively identifies relevant ranks consistent with the underlying structure of the data, outperforming traditional methods in scenarios where they are computationally infeasible or less accurate. This approach provides a more scalable and generalizable solution for rank determination in NMF that does not rely on domain-specific knowledge or assumptions.