Conditional density estimation in a regression setting

TL;DR

This paper develops a comprehensive theory for minimax estimation of the conditional density in regression, covering fixed and random designs, and introduces an adaptive estimator that achieves optimal performance across various settings.

Contribution

It introduces the first minimax theory for conditional density estimation in regression with fixed and random designs, and proposes a universal adaptive estimator.

Findings

Estimator matches oracle performance

Achieves sharp minimax rates over anisotropic classes

Is adaptive to design type (fixed or random)

Abstract

Regression problems are traditionally analyzed via univariate characteristics like the regression function, scale function and marginal density of regression errors. These characteristics are useful and informative whenever the association between the predictor and the response is relatively simple. More detailed information about the association can be provided by the conditional density of the response given the predictor. For the first time in the literature, this article develops the theory of minimax estimation of the conditional density for regression settings with fixed and random designs of predictors, bounded and unbounded responses and a vast set of anisotropic classes of conditional densities. The study of fixed design regression is of special interest and novelty because the known literature is devoted to the case of random predictors. For the aforementioned models, the paper suggests a universal adaptive estimator which (i) matches performance of an oracle that knows both an underlying model and an estimated conditional density; (ii) is sharp minimax over a vast class of anisotropic conditional densities; (iii) is at least rate minimax when the response is independent of the predictor and thus a bivariate conditional density becomes a univariate density; (iv) is adaptive to an underlying design (fixed or random) of predictors.