Lower bounds and aggregation in density estimation

TL;DR

This paper establishes the optimality of an aggregation method for density estimation by deriving lower bounds and demonstrating that the proposed estimator achieves these bounds, especially for the KL divergence.

Contribution

It proves lower bounds for aggregation of multiple density estimators and shows that a specific online estimator attains the optimal rate of convergence.

Findings

The lower bounds are established for KL, Hellinger, and L1 distances.

The proposed aggregation method achieves the optimal rate of $rac{ ext{log} M}{n}$.

The results confirm the optimality of an existing online estimator for density aggregation.

Abstract

In this paper we prove the optimality of an aggregation procedure. We prove lower bounds for aggregation of model selection type of $M$ density estimators for the Kullback-Leiber divergence (KL), the Hellinger's distance and the $L\_1$-distance. The lower bound, with respect to the KL distance, can be achieved by the on-line type estimate suggested, among others, by Yang (2000). Combining these results, we state that $\log M/n$ is an optimal rate of aggregation in the sense of Tsybakov (2003), where $n$ is the sample size.