Minimum L2 and robust Kullback-Leibler estimation

TL;DR

This paper proposes two new robust parameter estimation methods, minimum weighted L2 and robust Kullback-Leibler, which improve robustness and efficiency over traditional approaches, especially in normal models.

Contribution

It introduces two novel robust estimation techniques, providing influence functions and asymptotic variances, with applications to normal models and connections to local likelihood methods.

Findings

Methods are robust against model deviations

Asymptotic variances are derived and computed

Efficiency under normal model is demonstrated

Abstract

This paper introduces two new robust methods for estimation of parameters in a given parametric family. The first method is that of `minimum weighted L2', effectively minimising an estimate of the integrated (and possibly weighted) squared error. The second is `robust Kullback-Leibler', consisting of minimising a robust version of the empirical Kullback-Leibler distance, and can be viewed as a general robust modification of the maximum likelihood procedure. This second method is also related to recent local likelihood ideas for semiparametric density estimation. The methods are described, influence functions are found, as are formulae for asymptotic variances. In particular large-sample efficiencies are computed under the home turf conditions of the underlying parametric model. The methods and formulae are illustrated for the normal model.