ML-PWS: Estimating the Mutual Information Between Experimental Time Series Using Neural Networks

TL;DR

ML-PWS combines machine learning with Path Weight Sampling to accurately estimate the mutual information rate from experimental time series without requiring explicit system models.

Contribution

The paper introduces ML-PWS, a novel method that uses neural networks to create generative models from data, enabling exact information rate estimation in complex systems.

Findings

ML-PWS accurately estimates mutual information on synthetic data.

It outperforms traditional methods that require explicit models.

Successfully applied to neuronal time-series data.

Abstract

The ability to quantify information transmission is crucial for the analysis and design of natural and engineered systems. The information transmission rate is the fundamental measure for systems with time-varying signals, yet computing it is extremely challenging. In particular, the rate cannot be obtained directly from experimental time-series data without approximations, because of the high dimensionality of the signal trajectory space. Path Weight Sampling (PWS) is a computational technique that makes it possible to obtain the information rate exactly for any stochastic system. However, it requires a mathematical model of the system of interest, be it described by a master equation or a set of differential equations. Here, we present a technique that employs Machine Learning (ML) to develop a generative model from experimental time-series data, which is then combined with PWS to obtain the information rate. We demonstrate the accuracy of this technique, called ML-PWS, by comparing its results on synthetic time-series data generated from a non-linear model against ground-truth results obtained by applying PWS directly to the same model. We illustrate the utility of ML-PWS by applying it to neuronal time-series data.