Deconvolutional Density Network: Modeling Free-Form Conditional Distributions

TL;DR

This paper introduces the Deconvolutional Density Network (DDN), a neural network framework that models free-form conditional distributions effectively, especially in data-scarce scenarios, outperforming existing density estimation methods.

Contribution

The paper proposes a novel deconvolution-based neural network approach for flexible, free-form conditional density estimation that handles data deficiency and high-dimensionality better than prior methods.

Findings

DDN outperforms competing density estimation methods on various tasks.

The deconvolutional approach effectively models complex distributions.

DDN is robust in low-data and high-dimensional settings.

Abstract

Conditional density estimation (CDE) is the task of estimating the probability of an event conditioned on some inputs. A neural network (NN) can also be used to compute the output distribution for continuous-domain, which can be viewed as an extension of regression task. Nevertheless, it is difficult to explicitly approximate a distribution without knowing the information of its general form a priori. In order to fit an arbitrary conditional distribution, discretizing the continuous domain into bins is an effective strategy, as long as we have sufficiently narrow bins and very large data. However, collecting enough data is often hard to reach and falls far short of that ideal in many circumstances, especially in multivariate CDE for the curse of dimensionality. In this paper, we demonstrate the benefits of modeling free-form conditional distributions using a deconvolution-based neural net framework, coping with data deficiency problems in discretization. It has the advantage of being flexible but also takes advantage of the hierarchical smoothness offered by the deconvolution layers. We compare our method to a number of other density-estimation approaches and show that our Deconvolutional Density Network (DDN) outperforms the competing methods on many univariate and multivariate tasks. The code of DDN is available at https://github.com/NBICLAB/DDN.