An improved estimator of Shannon entropy with applications to systems with memory

TL;DR

This paper introduces an improved Shannon entropy estimator capable of accurately assessing memory in systems with correlations, demonstrated on precipitation data, and effective even with limited sampling.

Contribution

It presents a novel entropy estimator that accurately measures system memory in correlated and undersampled data, advancing previous methods.

Findings

Reliable memory detection in Markov systems

Accurate entropy estimation with correlations

Consistent results on precipitation data

Abstract

We investigate the memory properties of discrete sequences built upon a finite number of states. We find that the block entropy can reliably determine the memory for systems modeled as Markov chains of arbitrary finite order. Further, we provide an entropy estimator that remarkably gives accurate results when correlations are present. To illustrate our findings, we calculate the memory of daily precipitation series at different locations. Our results are in agreement with existing methods being at the same time valid in the undersampled regime and independent of model selection.