Characterising information-theoretic storage and transfer in continuous time processes

TL;DR

This paper extends information dynamics to continuous time processes, decomposing information storage into active memory utilization and instantaneous predictive capacity, with applications to neural spiking models.

Contribution

It provides a comprehensive formulation of information processing in continuous time, including new measures for memory utilization and insights into neural spiking processes.

Findings

Active information storage diverges, preventing a convergent predictive capacity rate.

Active memory utilization decomposes into jump and continuous components.

Application to neural models reveals discontinuous and continuous information contributions.

Abstract

The characterisation of information processing is an important task in complex systems science. Information dynamics is a quantitative methodology for modelling the intrinsic information processing conducted by a process represented as a time series, but to date has only been formulated in discrete time. Building on previous work which demonstrated how to formulate transfer entropy in continuous time, we give a total account of information processing in this setting, incorporating information storage. We find that a convergent rate of predictive capacity, comprised of the transfer entropy and active information storage, does not exist, arising through divergent rates of active information storage. We identify that active information storage can be decomposed into two separate quantities that characterise predictive capacity stored in a process: active memory utilisation and instantaneous predictive capacity. The latter involves prediction related to path regularity and so solely inherits the divergent properties of the active information storage, whilst the former permits definitions of pathwise and rate quantities. We formulate measures of memory utilisation for jump and neural spiking processes and illustrate measures of information processing in synthetic neural spiking models and coupled Ornstein-Uhlenbeck models. The application to synthetic neural spiking models demonstrates that active memory utilisation for point processes consists of discontinuous jump contributions (at spikes) interrupting a continuously varying contribution (relating to waiting times between spikes), complementing the behaviour previously demonstrated for transfer entropy in these processes.