Unintended Consequences: Updating Causal Models

TL;DR

This paper explores how agents update their causal beliefs based on feedback from their choices, integrating causal modeling with decision-making to understand steady states of belief and action.

Contribution

It introduces a framework combining probabilistic causal beliefs with agency and utility models to analyze belief updates and steady states.

Findings

Model of belief updating in causal structures

Analysis of feedback effects on causal beliefs

Characterization of steady states in causal decision-making

Abstract

We examine how causal beliefs affect an agent's choices and how feedback on those choices leads to updated causal beliefs. Building on the structural-equations framework for modeling causality, we first examine the general problem of updating causal beliefs in the face of novel (and possibly inexplicable) data. We model an agent who is uncertain of the true causal model, and therefore entertains a probabilistic belief over the set of possible models. We then consider how causal beliefs influence choices by building a model of agency and utility on top of the usual structural-equations framework. Using these two components, we propose a notion of steady state, where the feedback received from an agent's optimal action, given her current beliefs about the true causal model, can be rationalized by those beliefs.