The Elusive Present: Hidden Past and Future Dependency and Why We Build Models

TL;DR

This paper explores why Markovian models often fail to capture dependencies in temporal processes, showing that the present can hide past-future information and advocating for state-based models.

Contribution

It introduces a measure of elusive information, provides methods to calculate it, and explains the necessity of state-based models when the present conceals past-future dependencies.

Findings

Markovian assumptions often fail in structured processes.

The present can hide dependencies between past and future.

State-based models are necessary when the present conceals past-future information.

Abstract

Modeling a temporal process as if it is Markovian assumes the present encodes all of the process's history. When this occurs, the present captures all of the dependency between past and future. We recently showed that if one randomly samples in the space of structured processes, this is almost never the case. So, how does the Markov failure come about? That is, how do individual measurements fail to encode the past? And, how many are needed to capture dependencies between the past and future? Here, we investigate how much information can be shared between the past and future, but not be reflected in the present. We quantify this elusive information, give explicit calculational methods, and draw out the consequences. The most important of which is that when the present hides past-future dependency we must move beyond sequence-based statistics and build state-based models.