Deep Nonnegative Matrix Factorization with Beta Divergences

TL;DR

This paper introduces deep NMF models using beta-divergences, especially Kullback-Leibler divergence, to improve feature extraction in diverse data types like images, documents, and signals.

Contribution

It develops new deep NMF algorithms based on beta-divergences, extending beyond traditional least squares error approaches.

Findings

Enhanced facial feature extraction

Improved topic identification in documents

Effective material detection in hyperspectral images

Abstract

Deep Nonnegative Matrix Factorization (deep NMF) has recently emerged as a valuable technique for extracting multiple layers of features across different scales. However, all existing deep NMF models and algorithms have primarily centered their evaluation on the least squares error, which may not be the most appropriate metric for assessing the quality of approximations on diverse datasets. For instance, when dealing with data types such as audio signals and documents, it is widely acknowledged that $\beta$-divergences offer a more suitable alternative. In this paper, we develop new models and algorithms for deep NMF using some $\beta$-divergences, with a focus on the Kullback-Leibler divergence. Subsequently, we apply these techniques to the extraction of facial features, the identification of topics within document collections, and the identification of materials within hyperspectral images.