Memory Order Decomposition of Symbolic Sequences

TL;DR

This paper presents a novel method for analyzing memory in symbolic sequences using higher-order Markov models, allowing for accurate identification of correlation structures in both synthetic and real data.

Contribution

It introduces a new approach to decompose Markov processes into minimal order matrices, defining a clear memory profile for symbolic sequences.

Findings

Successfully recovers memory profiles of synthetic sequences.

Effectively extracts stochastic properties from real data.

Provides a practical protocol for sequence analysis.

Abstract

We introduce a general method for the study of memory in symbolic sequences based on higher-order Markov analysis. The Markov process that best represents a sequence is expressed as a mixture of matrices of minimal orders, enabling the definition of the so-called memory profile, which unambiguously reflects the true order of correlations. The method is validated by recovering the memory profiles of tunable synthetic sequences. Finally, we scan real data and showcase with practical examples how our protocol can be used to extract relevant stochastic properties of symbolic sequences.