Measuring Sample Path Causal Influences with Relative Entropy

TL;DR

This paper introduces a new sample path dependent causal influence measure for time series, leveraging sequential prediction theory, and demonstrates its advantages over existing measures like directed information through theoretical analysis and practical applications.

Contribution

It proposes a novel causal influence measure based on sample paths, along with a sequential prediction-based estimation method and finite sample regret bounds, addressing limitations of existing measures.

Findings

The new measure identifies specific influence patterns missed by directed information.

The proposed estimator's performance is bounded by a finite sample regret.

Application to stock market data demonstrates practical utility.

Abstract

We present a sample path dependent measure of causal influence between time series. The proposed causal measure is a random sequence, a realization of which enables identification of specific patterns that give rise to high levels of causal influence. We show that these patterns cannot be identified by existing measures such as directed information (DI). We demonstrate how sequential prediction theory may be leveraged to estimate the proposed causal measure and introduce a notion of regret for assessing the performance of such estimators. We prove a finite sample bound on this regret that is determined by the worst case regret of the sequential predictors used in the estimator. Justification for the proposed measure is provided through a series of examples, simulations, and application to stock market data. Within the context of estimating DI, we show that, because joint Markovicity of a pair of processes does not imply the marginal Markovicity of individual processes, commonly used plug-in estimators of DI will be biased for a large subset of jointly Markov processes. We introduce a notion of DI with "stale history", which can be combined with a plug-in estimator to upper and lower bound the DI when marginal Markovicity does not hold.