CAIMAN: Causal Action Influence Detection for Sample-efficient Loco-manipulation

TL;DR

CAIMAN introduces a reinforcement learning framework that enhances legged robots' ability to perform object pushing by using causal action influence as an intrinsic motivation, leading to sample-efficient learning and successful real-world transfer.

Contribution

The paper proposes CAIMAN, a novel RL approach that leverages causal influence for intrinsic motivation, improving sample efficiency and transferability in loco-manipulation tasks.

Findings

CAIMAN achieves higher sample efficiency in simulation.

The method successfully transfers to real robots without fine-tuning.

It enables effective object pushing in unstructured environments.

Abstract

Enabling legged robots to perform non-prehensile loco-manipulation is crucial for enhancing their versatility. Learning behaviors such as whole-body object pushing often requires sophisticated planning strategies or extensive task-specific reward shaping, especially in unstructured environments. In this work, we present CAIMAN, a practical reinforcement learning framework that encourages the agent to gain control over other entities in the environment. CAIMAN leverages causal action influence as an intrinsic motivation objective, allowing legged robots to efficiently acquire object pushing skills even under sparse task rewards. We employ a hierarchical control strategy, combining a low-level locomotion module with a high-level policy that generates task-relevant velocity commands and is trained to maximize the intrinsic reward. To estimate causal action influence, we learn the dynamics of the environment by integrating a kinematic prior with data collected during training. We empirically demonstrate CAIMAN's superior sample efficiency and adaptability to diverse scenarios in simulation, as well as its successful transfer to real-world systems without further fine-tuning. A video demo is available at https://www.youtube.com/watch?v=dNyvT04Cqaw.