Model selection for density estimation with L2-loss

TL;DR

This paper introduces a general model selection method for density estimation in L2-loss, capable of handling various model complexities and density bounds, with applications to adaptive estimation and aggregation.

Contribution

It provides a fully general, theoretical model selection framework for density estimation in L2, accommodating non-linear models and unknown density bounds.

Findings

Applicable to unbounded and bounded densities with unknown bounds

Enables adaptive estimation strategies

Facilitates aggregation of preliminary estimators

Abstract

We consider here estimation of an unknown probability density s belonging to L2(mu) where mu is a probability measure. We have at hand n i.i.d. observations with density s and use the squared L2-norm as our loss function. The purpose of this paper is to provide an abstract but completely general method for estimating s by model selection, allowing to handle arbitrary families of finite-dimensional (possibly non-linear) models and any density s belonging to L2(mu). We shall, in particular, consider the cases of unbounded densities and bounded densities with unknown bound and investigate how the L-infinity-norm of s may influence the risk. We shall also provide applications to adaptive estimation and aggregation of preliminary estimators. Although of a purely theoretical nature, our method leads to results that cannot presently be reached by more concrete methods.