Adaptive Bayesian Estimation of Conditional Densities

TL;DR

This paper introduces an adaptive Bayesian non-parametric model for estimating conditional densities using finite mixtures of normals with covariate-dependent weights, achieving optimal convergence rates without prior smoothness knowledge.

Contribution

It develops a novel Bayesian mixture model for conditional densities that adapts to unknown smoothness levels and irrelevant covariates, with proven asymptotic optimality.

Findings

Posterior contraction rates match minimax rates up to a log factor.

Model performs favorably compared to kernel estimators in simulations.

Rates are robust to irrelevant covariates inclusion.

Abstract

We consider a non-parametric Bayesian model for conditional densities. The model is a finite mixture of normal distributions with covariate dependent multinomial logit mixing probabilities. A prior for the number of mixture components is specified on positive integers. The marginal distribution of covariates is not modeled. We study asymptotic frequentist behavior of the posterior in this model. Specifically, we show that when the true conditional density has a certain smoothness level, then the posterior contraction rate around the truth is equal up to a log factor to the frequentist minimax rate of estimation. An extension to the case when the covariate space is unbounded is also established. As our result holds without a priori knowledge of the smoothness level of the true density, the established posterior contraction rates are adaptive. Moreover, we show that the rate is not affected by inclusion of irrelevant covariates in the model. In Monte Carlo simulations, a version of the model compares favorably to a cross-validated kernel conditional density estimator.