Neural-g: A Deep Learning Framework for Mixing Density Estimation

TL;DR

Neural-g is a flexible neural network-based framework for mixing density estimation that accurately captures complex prior distributions, including those with flat regions, heavy tails, and discontinuities, outperforming existing methods.

Contribution

We introduce neural-g, a novel neural network estimator for g-modeling that guarantees valid probability densities and demonstrates universal approximation capabilities for arbitrary prior shapes.

Findings

Neural-g effectively captures complex prior densities in simulations.

It outperforms existing methods in modeling flat, heavy-tailed, and discontinuous densities.

The approach extends to multivariate prior density estimation.

Abstract

Mixing (or prior) density estimation is an important problem in machine learning and statistics, especially in empirical Bayes $g$-modeling where accurately estimating the prior is necessary for making good posterior inferences. In this paper, we propose neural-$g$, a new neural network-based estimator for $g$-modeling. Neural-$g$ uses a softmax output layer to ensure that the estimated prior is a valid probability density. Under default hyperparameters, we show that neural-$g$ is very flexible and capable of capturing many unknown densities, including those with flat regions, heavy tails, and/or discontinuities. In contrast, existing methods struggle to capture all of these prior shapes. We provide justification for neural-$g$ by establishing a new universal approximation theorem regarding the capability of neural networks to learn arbitrary probability mass functions. To accelerate convergence of our numerical implementation, we utilize a weighted average gradient descent approach to update the network parameters. Finally, we extend neural-$g$ to multivariate prior density estimation. We illustrate the efficacy of our approach through simulations and analyses of real datasets. A software package to implement neural-$g$ is publicly available at https://github.com/shijiew97/neuralG.