Bayesian Cram\'er-Rao Bound Estimation with Score-Based Models

TL;DR

This paper introduces a data-driven estimator for the Bayesian Cramér-Rao bound using score matching, providing theoretical bounds and demonstrating applications in signal denoising and communication systems.

Contribution

It develops a novel score matching-based estimator for the Bayesian CRB and analyzes its performance with theoretical bounds in parametric and neural network regimes.

Findings

The estimator performs well in signal denoising tasks.

It provides accurate bounds in neural network modeling.

Theoretical non-asymptotic bounds are established for score matching errors.

Abstract

The Bayesian Cram\'er-Rao bound (CRB) provides a lower bound on the mean square error of any Bayesian estimator under mild regularity conditions. It can be used to benchmark the performance of statistical estimators, and provides a principled metric for system design and optimization. However, the Bayesian CRB depends on the underlying prior distribution, which is often unknown for many problems of interest. This work introduces a new data-driven estimator for the Bayesian CRB using score matching, i.e., a statistical estimation technique that models the gradient of a probability distribution from a given set of training data. The performance of the proposed estimator is analyzed in both the classical parametric modeling regime and the neural network modeling regime. In both settings, we develop novel non-asymptotic bounds on the score matching error and our Bayesian CRB estimator based on the results from empirical process theory, including classical bounds and recently introduced techniques for characterizing neural networks. We illustrate the performance of the proposed estimator with two application examples: a signal denoising problem and a dynamic phase offset estimation problem in communication systems.