Information in additional observations of a non-parametric experiment that is not estimable

TL;DR

This paper investigates the diminishing informational value of additional observations in certain non-parametric experiments, showing it decreases at a rate of 1/√n, unlike the typical 1/n rate in parametric cases.

Contribution

It demonstrates that in some non-parametric experiments, the value of extra data diminishes at a slower rate of 1/√n, and that no consistent estimator exists despite the value tending to zero.

Findings

Value of additional observations decreases at rate 1/√n in certain non-parametric experiments.

In some cases, the value tends to zero without the existence of a consistent estimator.

The rate differs from the typical 1/n decrease in parametric experiments.

Abstract

Given $n$ independent and identically distributed observations and measuring the value of obtaining an additional observation in terms of Le Cam's notion of deficiency between experiments, we show for certain types of non-parametric experiments that the value of an additional observation decreases at a rate of $1/\sqrt{n}$. This is distinct from the known typical decrease at a rate of $1/n$ for parametric experiments and the non-decreasing value in the case of very large experiments. In particular, the rate of $1/\sqrt{n}$ holds for the experiment given by observing samples from a density about which we know only that it is bounded from below by some fixed constant. Thus there exists an experiment where the value of additional observations tends to zero but for which no estimator that is consistent (in total variation distance) exists.