Empirical Bayes conditional density estimation

TL;DR

This paper develops an empirical Bayes method for nonparametric conditional density estimation using Gaussian mixtures, achieving minimax-optimal rates and adaptive dimension reduction, applicable across various fields.

Contribution

It introduces a data-driven empirical Bayes approach with predictor-dependent Gaussian mixture priors, providing adaptive, minimax-optimal conditional density estimators.

Findings

Achieves near minimax-optimal convergence rates.

Performs automatic dimension reduction when some predictors are irrelevant.

Applicable to diverse fields like economics, actuarial sciences, and medicine.

Abstract

The problem of nonparametric estimation of the conditional density of a response, given a vector of explanatory variables, is classical and of prominent importance in many prediction problems since the conditional density provides a more comprehensive description of the association between the response and the predictor than, for instance, does the regression function. The problem has applications across different fields like economy, actuarial sciences and medicine. We investigate empirical Bayes estimation of conditional densities establishing that an automatic data-driven selection of the prior hyper-parameters in infinite mixtures of Gaussian kernels, with predictor-dependent mixing weights, can lead to estimators whose performance is on par with that of frequentist estimators in being minimax-optimal (up to logarithmic factors) rate adaptive over classes of locally H\"older smooth conditional densities and in performing an adaptive dimension reduction if the response is independent of (some of) the explanatory variables which, containing no information about the response, are irrelevant to the purpose of estimating its conditional density.