An Interpretation of the Moore-Penrose Generalized Inverse of a Singular Fisher Information Matrix

TL;DR

This paper demonstrates that the Moore-Penrose generalized inverse of a singular Fisher information matrix can be interpreted as a valid Cramer-Rao bound for minimum variance estimators under certain constraints, ensuring its proper use in singular cases.

Contribution

It provides a novel interpretation of the Moore-Penrose inverse of a singular FIM as a valid lower bound for estimator variance, extending the applicability of the CRB.

Findings

Moore-Penrose inverse of singular FIM acts as a CRB for constrained estimators.

Ensures logical validity of using generalized inverse as covariance lower bound.

Applicable to joint design of constraints and unbiased estimators.

Abstract

It is proved that in a non-Bayesian parametric estimation problem, if the Fisher information matrix (FIM) is singular, unbiased estimators for the unknown parameter will not exist. Cramer-Rao bound (CRB), a popular tool to lower bound the variances of unbiased estimators, seems inapplicable in such situations. In this paper, we show that the Moore-Penrose generalized inverse of a singular FIM can be interpreted as the CRB corresponding to the minimum variance among all choices of minimum constraint functions. This result ensures the logical validity of applying the Moore-Penrose generalized inverse of an FIM as the covariance lower bound when the FIM is singular. Furthermore, the result can be applied as a performance bound on the joint design of constraint functions and unbiased estimators.