Revisiting Le Cam's Equation: Exact Minimax Rates over Convex Density Classes

TL;DR

This paper derives exact minimax rates for density estimation over convex classes, extending classical results and introducing a multistage sieve MLE that adapts to various convex density classes.

Contribution

It provides the first exact minimax rates over convex density classes, linking local metric entropy to optimal rates, and introduces an adaptive multistage sieve MLE applicable to these classes.

Findings

Exact minimax rates over convex density classes established.

Multistage sieve MLE is adaptive and applicable to various classes.

Re-derivation of known rates and new bounds for complex classes.

Abstract

We study the classical problem of deriving minimax rates for density estimation over convex density classes. Building on the pioneering work of Le Cam (1973), Birge (1983, 1986), Wong and Shen (1995), Yang and Barron (1999), we determine the exact (up to constants) minimax rate over any convex density class. This work thus extends these known results by demonstrating that the local metric entropy of the density class always captures the minimax optimal rates under such settings. Our bounds provide a unifying perspective across both parametric and nonparametric convex density classes, under weaker assumptions on the richness of the density class than previously considered. Our proposed `multistage sieve' MLE applies to any such convex density class. We further demonstrate that this estimator is also adaptive to the true underlying density of interest. We apply our risk bounds to rederive known minimax rates including bounded total variation, and Holder density classes. We further illustrate the utility of the result by deriving upper bounds for less studied classes, e.g., convex mixture of densities.