Which bits went where? Past and future transfer entropy decomposition with the information bottleneck

TL;DR

This paper introduces a novel method to decompose transfer entropy using the information bottleneck, allowing for detailed analysis of information flow between processes in complex systems.

Contribution

It presents a new approach to decompose transfer entropy into past and future components using the information bottleneck, enhancing understanding of information dynamics.

Findings

Successfully applied to synthetic recurrent processes

Effectively decomposed transfer entropy in neural data

Revealed nuanced information flow dynamics

Abstract

Whether the system under study is a shoal of fish, a collection of neurons, or a set of interacting atmospheric and oceanic processes, transfer entropy measures the flow of information between time series and can detect possible causal relationships. Much like mutual information, transfer entropy is generally reported as a single value summarizing an amount of shared variation, yet a more fine-grained accounting might illuminate much about the processes under study. Here we propose to decompose transfer entropy and localize the bits of variation on both sides of information flow: that of the originating process's past and that of the receiving process's future. We employ the information bottleneck (IB) to compress the time series and identify the transferred entropy. We apply our method to decompose the transfer entropy in several synthetic recurrent processes and an experimental mouse dataset of concurrent behavioral and neural activity. Our approach highlights the nuanced dynamics within information flow, laying a foundation for future explorations into the intricate interplay of temporal processes in complex systems.