Causal Imitation Learning under Expert-Observable and Expert-Unobservable Confounding

TL;DR

This paper introduces a causal imitation learning framework that handles hidden confounders using trajectory histories as instruments, and presents an algorithm that outperforms existing methods in continuous environments.

Contribution

It develops a unified causal IL framework with hidden confounders and proposes DML-IL, an instrumental variable regression-based algorithm, with theoretical bounds and empirical success.

Findings

DML-IL outperforms existing causal IL baselines in Mujoco tasks.

The framework effectively handles two types of hidden confounders.

The approach reformulates causal IL as a Conditional Moment Restriction problem.

Abstract

We propose a general framework for causal Imitation Learning (IL) with hidden confounders, which subsumes several existing settings. Our framework accounts for two types of hidden confounders: (a) variables observed by the expert but not by the imitator, and (b) confounding noise hidden from both. By leveraging trajectory histories as instruments, we reformulate causal IL in our framework into a Conditional Moment Restriction (CMR) problem. We propose DML-IL, an algorithm that solves this CMR problem via instrumental variable regression, and upper bound its imitation gap. Empirical evaluation on continuous state-action environments, including Mujoco tasks, demonstrates that DML-IL outperforms existing causal IL baselines.