Efficient functional estimation and the super-oracle phenomenon

TL;DR

This paper introduces an efficient weighted nearest neighbor estimator for two-sample integral functionals, achieving minimax optimality and revealing that in some cases it outperforms the ideal oracle estimator.

Contribution

It establishes the efficiency of a weighted nearest neighbor estimator for integral functionals and uncovers the super-oracle phenomenon where it surpasses the oracle estimator in certain scenarios.

Findings

Estimator achieves local asymptotic minimax lower bound.

Central limit theorem for the estimator enables valid confidence intervals.

In some cases, the estimator outperforms the oracle estimator.

Abstract

We consider the estimation of two-sample integral functionals, of the type that occur naturally, for example, when the object of interest is a divergence between unknown probability densities. Our first main result is that, in wide generality, a weighted nearest neighbour estimator is efficient, in the sense of achieving the local asymptotic minimax lower bound. Moreover, we also prove a corresponding central limit theorem, which facilitates the construction of asymptotically valid confidence intervals for the functional, having asymptotically minimal width. One interesting consequence of our results is the discovery that, for certain functionals, the worst-case performance of our estimator may improve on that of the natural `oracle' estimator, which is given access to the values of the unknown densities at the observations.