Active Causal Experimentalist (ACE): Learning Intervention Strategies via Direct Preference Optimization

TL;DR

ACE is a novel method that learns adaptive intervention strategies for causal discovery by optimizing pairwise preferences, outperforming traditional methods and discovering principled strategies through experience.

Contribution

It introduces Direct Preference Optimization for learning causal intervention policies from pairwise comparisons, enabling adaptive and principled experimental design.

Findings

Achieves 70-71% improvement over baselines at equal intervention budgets

Learns to target collider mechanisms with concentrated interventions

Demonstrates effectiveness across synthetic, physics, and economic datasets

Abstract

Discovering causal relationships requires controlled experiments, but experimentalists face a sequential decision problem: each intervention reveals information that should inform what to try next. Traditional approaches such as random sampling, greedy information maximization, and round-robin coverage treat each decision in isolation, unable to learn adaptive strategies from experience. We propose Active Causal Experimentalist (ACE), which learns experimental design as a sequential policy. Our key insight is that while absolute information gains diminish as knowledge accumulates (making value-based RL unstable), relative comparisons between candidate interventions remain meaningful throughout. ACE exploits this via Direct Preference Optimization, learning from pairwise intervention comparisons rather than non-stationary reward magnitudes. Across synthetic benchmarks, physics simulations, and economic data, ACE achieves 70-71% improvement over baselines at equal intervention budgets (p < 0.001, Cohen's d ~ 2). Notably, the learned policy autonomously discovers that collider mechanisms require concentrated interventions on parent variables, a theoretically-grounded strategy that emerges purely from experience. This suggests preference-based learning can recover principled experimental strategies, complementing theory with learned domain adaptation.