Improving Generalization Ability of Robotic Imitation Learning by Resolving Causal Confusion in Observations

TL;DR

This paper introduces a causal structure learning framework to improve the generalization of robotic imitation learning, effectively handling domain shifts by disentangling relevant observation features from irrelevant ones.

Contribution

It proposes a simple, easily integrable causal structure learning method that enhances generalization in complex imitation learning tasks without requiring disentangled feature representations.

Findings

Significantly reduces generalization errors in robotic imitation learning

Successfully applied to Mujoco simulation with bimanual robot arms

Improves robustness against domain shifts in imitation tasks

Abstract

Recent developments in imitation learning have considerably advanced robotic manipulation. However, current techniques in imitation learning can suffer from poor generalization, limiting performance even under relatively minor domain shifts. In this work, we aim to enhance the generalization capabilities of complex imitation learning algorithms to handle unpredictable changes from the training environments to deployment environments. To avoid confusion caused by observations that are not relevant to the target task, we propose to explicitly learn the causal relationship between observation components and expert actions, employing a framework similar to [6], where a causal structural function is learned by intervention on the imitation learning policy. Disentangling the feature representation from image input as in [6] is hard to satisfy in complex imitation learning process in robotic manipulation, we theoretically clarify that this requirement is not necessary in causal relationship learning. Therefore, we propose a simple causal structure learning framework that can be easily embedded in recent imitation learning architectures, such as the Action Chunking Transformer [31]. We demonstrate our approach using a simulation of the ALOHA [31] bimanual robot arms in Mujoco, and show that the method can considerably mitigate the generalization problem of existing complex imitation learning algorithms.