Minimax Optimal Conditional Density Estimation under Total Variation Smoothness

TL;DR

This paper establishes the minimax rates for nonparametric conditional density estimation under total variation smoothness in a multivariate context, proposing adaptive and non-adaptive kernel-based estimators that do not assume knowledge of smoothness parameters.

Contribution

It introduces the first minimax optimal estimators for conditional density under total variation smoothness, including an adaptive method that does not require prior smoothness knowledge.

Findings

Proves impossibility of estimation with only x-smoothness assumptions.

Proposes kernel-based estimators achieving minimax rates.

Develops an adaptive estimator without smoothness parameter knowledge.

Abstract

This paper studies the minimax rate of nonparametric conditional density estimation under a weighted absolute value loss function in a multivariate setting. We first demonstrate that conditional density estimation is impossible if one only requires that $p_{X|Z}$ is smooth in $x$ for all values of $z$. This motivates us to consider a sub-class of absolutely continuous distributions, restricting the conditional density $p_{X|Z}(x|z)$ to not only be H\"older smooth in $x$, but also be total variation smooth in $z$. We propose a corresponding kernel-based estimator and prove that it achieves the minimax rate. We give some simple examples of densities satisfying our assumptions which imply that our results are not vacuous. Finally, we propose an estimator which achieves the minimax optimal rate adaptively, i.e., without the need to know the smoothness parameter values in advance. Crucially, both of our estimators (the adaptive and non-adaptive ones) impose no assumptions on the marginal density $p_Z$, and are not obtained as a ratio between two kernel smoothing estimators which may sound like a go to approach in this problem.