Estimating the conditional density by histogram type estimators and model selection

TL;DR

This paper introduces a new data-driven method for estimating conditional densities using histogram-like estimators, providing non-asymptotic guarantees and adaptive convergence rates for complex function spaces.

Contribution

It develops a novel model selection procedure with oracle inequalities for conditional density estimation, applicable to a wide range of function classes and minimal assumptions.

Findings

Achieves adaptive convergence rates over Besov spaces.

Provides non-asymptotic oracle inequalities for selected estimators.

Ensures estimators have desirable statistical properties under mild conditions.

Abstract

We propose a new estimation procedure of the conditional density for independent and identically distributed data. Our procedure aims at using the data to select a function among arbitrary (at most countable) collections of candidates. By using a deterministic Hellinger distance as loss, we prove that the selected function satisfies a non-asymptotic oracle type inequality under minimal assumptions on the statistical setting. We derive an adaptive piecewise constant estimator on a random partition that achieves the expected rate of convergence over (possibly inhomogeneous and anisotropic) Besov spaces of small regularity. Moreover, we show that this oracle inequality may lead to a general model selection theorem under very mild assumptions on the statistical setting. This theorem guarantees the existence of estimators possessing nice statistical properties under various assumptions on the conditional density (such as smoothness or structural ones).