A variational autoencoder-based nonnegative matrix factorisation model for deep dictionary learning

TL;DR

This paper introduces a novel probabilistic model combining variational autoencoders with nonnegative matrix factorization to learn robust, nonnegative dictionaries for signal processing tasks, outperforming existing methods.

Contribution

It proposes a new VAE-based NMF model with a Gamma-distributed latent space and a specialized loss function for nonnegativity, advancing deep dictionary learning techniques.

Findings

VAE-NMF outperforms state-of-the-art methods in dictionary learning.

The model effectively enhances speech signals.

The approach successfully extracts muscle synergies.

Abstract

Construction of dictionaries using nonnegative matrix factorisation (NMF) has extensive applications in signal processing and machine learning. With the advances in deep learning, training compact and robust dictionaries using deep neural networks, i.e., dictionaries of deep features, has been proposed. In this study, we propose a probabilistic generative model which employs a variational autoencoder (VAE) to perform nonnegative dictionary learning. In contrast to the existing VAE models, we cast the model under a statistical framework with latent variables obeying a Gamma distribution and design a new loss function to guarantee the nonnegative dictionaries. We adopt an acceptance-rejection sampling reparameterization trick to update the latent variables iteratively. We apply the dictionaries learned from VAE-NMF to two signal processing tasks, i.e., enhancement of speech and extraction of muscle synergies. Experimental results demonstrate that VAE-NMF performs better in learning the latent nonnegative dictionaries in comparison with state-of-the-art methods.