# Efficiency requires innovation

## TL;DR

The paper discusses the limitations of jackknife extensions in achieving efficiency for parameter estimation and emphasizes the need for innovative methods beyond simple extensions, especially for non-exponential family distributions.

## Contribution

It introduces the concept of jackknife extension and demonstrates its inefficiency for general distributions, highlighting the necessity for innovative estimation techniques.

## Key findings

- Jackknife extension improves variance reduction but lacks innovation.
- Asymptotic efficiency with jackknife extension is limited to exponential families.
- Innovative methods are essential for efficient estimation in general models.

## Abstract

In estimation a parameter $\theta\in{\mathbb R}$ from a sample $(x_1,\ldots,x_n)$ from a population $P_{\theta}$ a simple way of incorporating a new observation $x_{n+1}$ into an estimator $\tilde\theta_{n} = \tilde\theta_{n}(x_1,\ldots,x_n)$ is transforming $\tilde\theta_n$ to what we call the {\it jackknife extension} $\tilde\theta_{n+1}^{(e)} = \tilde\theta_{n+1}^{(e)}(x_1,\ldots,x_n,x_{n+1})$, \[\tilde\theta_{n+1}^{(e)} = \{\tilde\theta_n (x_1 ,\ldots,x_n)+ \tilde\theta_n (x_{n+1},x_2 ,\ldots,x_n) + \ldots + \tilde\theta_n (x_1 ,\ldots,x_{n-1},x_{n+1})\}/(n+1).\] Though $\tilde\theta_{n+1}^{(e)}$ lacks an innovation the statistician could expect from a larger data set, it is still better than $\tilde\theta_n$, \[{\rm var}(\tilde\theta_{n+1}^{(e)})\leq\frac{n}{n+1} {\rm var}(\tilde\theta_n).\] However, an estimator obtained by jackknife extension for all $n$ is asymptotically efficient only for samples from exponential families. For a general $P_{\theta}$, asymptotically efficient estimators require innovation when a new observation is added to the data. Some examples illustrate the concept.