Minimum description length as an objective function for non-negative matrix factorization

TL;DR

This paper introduces MDL-NMF, a novel objective function for non-negative matrix factorization that balances model complexity and accuracy automatically using the minimum description length principle, reducing the need for parameter tuning.

Contribution

The paper proposes MDL-NMF, a new formulation for NMF that inherently manages sparsity and complexity without ad-hoc constraints or extensive parameter tuning.

Findings

Effective on three diverse real-world datasets

Performs well across various semi-synthetic data scenarios

Automatically balances model complexity and accuracy

Abstract

Non-negative matrix factorization (NMF) is a dimensionality reduction technique which tends to produce a sparse representation of data. Commonly, the error between the actual and recreated matrices is used as an objective function, but this method may not produce the type of representation we desire as it allows for the complexity of the model to grow, constrained only by the size of the subspace and the non-negativity requirement. If additional constraints, such as sparsity, are imposed the question of parameter selection becomes critical. Instead of adding sparsity constraints in an ad-hoc manner we propose a novel objective function created by using the principle of minimum description length (MDL). Our formulation, MDL-NMF, automatically trades off between the complexity and accuracy of the model using a principled approach with little parameter selection or the need for domain expertise. We demonstrate our model works effectively on three heterogeneous data-sets and on a range of semi-synthetic data showing the broad applicability of our method.