Recurrent Estimation of Distributions

TL;DR

This paper introduces RED, a novel neural network-based method for density estimation of real-valued data that improves modeling efficiency and accuracy, especially in anomaly detection tasks.

Contribution

RED uniquely employs RNNs to transform inputs and compute conditional distributions, offering a semiparametric, efficient, and normalized density estimation approach.

Findings

RED achieves lower negative log-likelihood than other neural methods.

RED performs better in anomaly detection tasks.

RED is efficient to train, compute, and sample from.

Abstract

This paper presents the recurrent estimation of distributions (RED) for modeling real-valued data in a semiparametric fashion. RED models make two novel uses of recurrent neural networks (RNNs) for density estimation of general real-valued data. First, RNNs are used to transform input covariates into a latent space to better capture conditional dependencies in inputs. After, an RNN is used to compute the conditional distributions of the latent covariates. The resulting model is efficient to train, compute, and sample from, whilst producing normalized pdfs. The effectiveness of RED is shown via several real-world data experiments. Our results show that RED models achieve a lower held-out negative log-likelihood than other neural network approaches across multiple dataset sizes and dimensionalities. Further context of the efficacy of RED is provided by considering anomaly detection tasks, where we also observe better performance over alternative models.