Sequential Extensions of Causal and Evidential Decision Theory

TL;DR

This paper explores how causal and evidential decision theories can be extended to sequential decision-making, revealing that evidential theory has two natural extensions whereas causal theory has only one.

Contribution

It introduces sequential extensions of causal and evidential decision theories, clarifying their differences and natural extensions in dynamic settings.

Findings

Evidential decision theory admits two natural sequential extensions.

Causal decision theory has only one natural sequential extension.

The work clarifies the application of these theories in dynamic environments.

Abstract

Moving beyond the dualistic view in AI where agent and environment are separated incurs new challenges for decision making, as calculation of expected utility is no longer straightforward. The non-dualistic decision theory literature is split between causal decision theory and evidential decision theory. We extend these decision algorithms to the sequential setting where the agent alternates between taking actions and observing their consequences. We find that evidential decision theory has two natural extensions while causal decision theory only has one.