Dictionary Based Pattern Entropy for Causal Direction Discovery

TL;DR

This paper introduces a Dictionary Based Pattern Entropy framework that combines information theory to infer causal directions from symbolic sequences by analyzing pattern structures and their entropy, demonstrating reliable performance across synthetic and real datasets.

Contribution

The novel DPE framework integrates AIT and Shannon information theory to infer causality through pattern-based dictionaries and entropy measures, advancing causal discovery methods for symbolic data.

Findings

DPE outperforms or matches existing AIT-based methods on synthetic systems.

DPE shows competitive performance on biological and ecological datasets.

Minimizing pattern uncertainty provides a robust approach for causal inference.

Abstract

Discovering causal direction from temporal observational data is particularly challenging for symbolic sequences, where functional models and noise assumptions are often unavailable. We propose a novel \emph{Dictionary Based Pattern Entropy ($DPE$)} framework that infers both the direction of causation and the specific subpatterns driving changes in the effect variable. The framework integrates \emph{Algorithmic Information Theory} (AIT) and \emph{Shannon Information Theory}. Causation is interpreted as the emergence of compact, rule based patterns in the candidate cause that systematically constrain the effect. $DPE$ constructs direction-specific dictionaries and quantifies their influence using entropy-based measures, enabling a principled link between deterministic pattern structure and stochastic variability. Causal direction is inferred via a minimum-uncertainty criterion, selecting the direction exhibiting stronger and more consistent pattern-driven organization. As summarized in Table 7, $DPE$ consistently achieves reliable performance across diverse synthetic systems, including delayed bit-flip perturbations, AR(1) coupling, 1D skew-tent maps, and sparse processes, outperforming or matching competing AIT-based methods ($ETC_E$, $ETC_P$, $LZ_P$). In biological and ecological datasets, performance is competitive, while alternative methods show advantages in specific genomic settings. Overall, the results demonstrate that minimizing pattern level uncertainty yields a robust, interpretable, and broadly applicable framework for causal discovery.