Rates of convergence of rho-estimators for sets of densities satisfying shape constraints

TL;DR

This paper investigates the convergence rates of rho-estimators for shape-constrained density models, showing that near extremal points, the risk can be significantly lower than the worst-case minimax bounds, especially for specific models like decreasing densities.

Contribution

It introduces refined risk bounds for rho-estimators at extremal and near-extremal points, improving understanding of estimator performance beyond traditional minimax analysis.

Findings

Risk at extremal points is substantially smaller than the minimax risk.

Near extremal points, the risk can be bounded by the extremal risk plus a squared distance term.

Refined empirical process bounds underpin the improved risk analysis.

Abstract

The purpose of this paper is to pursue our study of rho-estimators built from i.i.d. observations that we defined in Baraud et al. (2014). For a \rho-estimator based on some model S (which means that the estimator belongs to S) and a true distribution of the observations that also belongs to S, the risk (with squared Hellinger loss) is bounded by a quantity which can be viewed as a dimension function of the model and is often related to the "metric dimension" of this model, as defined in Birg\'e (2006). This is a minimax point of view and it is well-known that it is pessimistic. Typically, the bound is accurate for most points in the model but may be very pessimistic when the true distribution belongs to some specific part of it. This is the situation that we want to investigate here. For some models, like the set of decreasing densities on [0,1], there exist specific points in the model that we shall call "extremal" and for which the risk is substantially smaller than the typical risk. Moreover, the risk at a non-extremal point of the model can be bounded by the sum of the risk bound at a well-chosen extremal point plus the square of its distance to this point. This implies that if the true density is close enough to an extremal point, the risk at this point may be smaller than the minimax risk on the model and this actually remains true even if the true density does not belong to the model. The result is based on some refined bounds on the suprema of empirical processes that are established in Baraud (2016).