Likelihood Functions with Parameter-Dependent Support: A Survey of the Cram\'{e}r-Rao-Leibniz Lower Bound

TL;DR

This paper reviews and generalizes the Cramér-Rao lower bound for parameter estimation to cases where the likelihood function's support depends on the parameter, introducing the CRLLB and unifying existing results.

Contribution

It introduces the CRLLB for parameter-dependent support, unifies existing bounds under a multidimensional framework, and surveys practical examples illustrating its application.

Findings

CRLLB extends CRLB to parameter-dependent support cases.

Unified framework for multidimensional parameters.

Illustrative examples demonstrate CRLLB's usefulness.

Abstract

Parameter estimation is a fundamental problem in science and engineering. In many safety-critical applications, one is not only interested in a {\it point} estimator, but also the uncertainty bound that can self-assess the accuracy of the estimator. In this regard, the Cram\'{e}r-Rao lower bound (CRLB) is of great importance, as it provides a lower bound on the variance of {\it any} unbiased estimator. In many cases, it is the only way of evaluating, without recourse to simulations, the expected accuracy of numerically obtainable estimates. For the existence of the CRLB, there have been widely accepted regularity conditions, one of which is that the support of the likelihood function (LF) -- the pdf of the observations conditioned on the parameter of interest -- should be independent of the parameter to be estimated. This paper starts from reviewing the derivations of the classical CRLB under the condition that the LF has parameter-independent support. To cope with the case of parameter-dependent support, we generalize the CRLB to the {\it Cram\'{e}r-Rao-Leibniz lower bound (CRLLB)}, by leveraging the general Leibniz integral rule. Notably, the existing results on CRLLB and CRLB are unified under the framework of CRLLB with multidimensional parameters. Then, we survey existing examples of LFs to illustrate the usefulness of the CRLLB in providing valid covariance bounds.