Empirical MSE Minimization to Estimate a Scalar Parameter

TL;DR

This paper introduces a method for estimating a scalar parameter by optimally combining two estimators, one consistent and one potentially inconsistent but with lower variance, to minimize mean-squared error.

Contribution

It proposes a new estimator that combines two existing estimators, achieving better worst-case MSE performance than either alone.

Findings

The combined estimator outperforms individual estimators in worst-case MSE.

The method guarantees a minimax-regret optimality.

The approach is applicable when one estimator is consistent and the other has lower variance.

Abstract

We consider the estimation of a scalar parameter, when two estimators are available. The first is always consistent. The second is inconsistent in general, but has a smaller asymptotic variance than the first, and may be consistent if an assumption is satisfied. We propose to use the weighted sum of the two estimators with the lowest estimated mean-squared error (MSE). We show that this third estimator dominates the other two from a minimax-regret perspective: the maximum asymptotic-MSE-gain one may incur by using this estimator rather than one of the other estimators is larger than the maximum asymptotic-MSE-loss.