InfoMat: Leveraging Information Theory to Visualize and Understand Sequential Data

TL;DR

InfoMat is a new tool that uses information theory to visualize and understand complex dependencies in sequential data, making it easier to analyze time series data.

Contribution

The novel information matrix (InfoMat) provides a structured visual representation of sequential information transfer.

Findings

InfoMat captures sequential information measures like directed information and transfer entropy.

Efficient estimators for InfoMat enable its application to real-world datasets.

InfoMat is useful in neuroscience, finance, communication systems, and machine learning.

Abstract

Despite the widespread use of information measures in analyzing probabilistic systems, effective visualization tools for understanding complex dependencies in sequential data are scarce. In this work, we introduce the information matrix (InfoMat), a novel and intuitive matrix representation of information transfer in sequential systems. InfoMat provides a structured visual perspective on mutual information decompositions, enabling the discovery of new relationships between sequential information measures and enhancing interpretability in time series data analytics. We demonstrate how InfoMat captures key sequential information measures, such as directed information and transfer entropy. To facilitate its application in real-world datasets, we propose both an efficient Gaussian mutual information estimator and a neural InfoMat estimator based on masked autoregressive flows to model more complex dependencies. These estimators make InfoMat a valuable tool for uncovering hidden patterns in data analytics applications, encompassing neuroscience, finance, communication systems, and machine learning. We further illustrate the utility of InfoMat in visualizing information flow in real-world sequential physiological data analysis and in visualizing information flow in communication channels under various coding schemes. By mapping visual patterns in InfoMat to various modes of dependence structures, we provide a data-driven framework for analyzing causal relationships and temporal interactions. InfoMat thus serves as both a theoretical and empirical tool for data-driven decision making, bridging the gap between information theory and applied data analytics.

Figures