Non-parametric estimation of Fisher information from real data

TL;DR

This paper introduces a non-parametric method to estimate Fisher information directly from real data, utilizing density estimation via field theory, and demonstrates its effectiveness through validation and application to the Ising model.

Contribution

The authors develop a novel non-parametric algorithm for estimating Fisher information from real data, including a numerical procedure to optimize finite difference intervals.

Findings

Accurately estimates Fisher information using density estimation with field theory.

Validates the method with normal distribution Fisher information.

Applies the method to the Ising model, identifying phase transition points.

Abstract

The Fisher Information matrix is a widely used measure for applications ranging from statistical inference, information geometry, experiment design, to the study of criticality in biological systems. Yet there is no commonly accepted non-parametric algorithm to estimate it from real data. In this rapid communication we show how to accurately estimate the Fisher information in a nonparametric way. We also develop a numerical procedure to minimize the errors by choosing the interval of the finite difference scheme necessary to compute the derivatives in the definition of the Fisher information. Our method uses the recently published "Density Estimation using Field Theory" algorithm to compute the probability density functions for continuous densities. We use the Fisher information of the normal distribution to validate our method and as an example we compute the temperature component of the Fisher Information Matrix in the two dimensional Ising model and show that it obeys the expected relation to the heat capacity and therefore peaks at the phase transition at the correct critical temperature.