The RKHS Approach to Minimum Variance Estimation Revisited: Variance Bounds, Sufficient Statistics, and Exponential Families

TL;DR

This paper advances the mathematical understanding of minimum variance estimation using RKHS by developing geometric bounds, analyzing the role of sufficient statistics, and deriving new bounds for exponential family distributions.

Contribution

It extends RKHS analysis of MVE by formulating variance bounds geometrically, analyzing their properties, and deriving new bounds for exponential family models.

Findings

Barankin bound is lower semicontinuous in parameters.

RKHS remains unchanged with sufficient statistics.

New closed-form bounds for exponential family distributions.

Abstract

The mathematical theory of reproducing kernel Hilbert spaces (RKHS) provides powerful tools for minimum variance estimation (MVE) problems. Here, we extend the classical RKHS based analysis of MVE in several directions. We develop a geometric formulation of five known lower bounds on the estimator variance (Barankin bound, Cramer-Rao bound, constrained Cramer-Rao bound, Bhattacharyya bound, and Hammersley-Chapman-Robbins bound) in terms of orthogonal projections onto a subspace of the RKHS associated with a given MVE problem. We show that, under mild conditions, the Barankin bound (the tightest possible lower bound on the estimator variance) is a lower semicontinuous function of the parameter vector. We also show that the RKHS associated with an MVE problem remains unchanged if the observation is replaced by a sufficient statistic. Finally, for MVE problems conforming to an exponential family of distributions, we derive novel closed-form lower bound on the estimator variance and show that a reduction of the parameter set leaves the minimum achievable variance unchanged.