RobotMover: Learning to Move Large Objects From Human Demonstrations

TL;DR

RobotMover is a learning-based system that enables robots to manipulate large objects by learning from human demonstrations, using a novel interaction representation to generalize across robot types and transfer to real-world tasks.

Contribution

Introduces RobotMover, a new imitation learning framework utilizing the Interaction Chain for generalizable large object manipulation across different robots.

Findings

RobotMover enables a Spot robot to manipulate various large objects.

The system outperforms baseline methods in robustness and controllability.

Zero-shot transfer from simulation to real-world robots is achieved.

Abstract

Moving large objects, such as furniture or appliances, is a critical capability for robots operating in human environments. This task presents unique challenges, including whole-body coordination to avoid collisions and managing the dynamics of bulky, heavy objects. In this work, we present RobotMover, a learning-based system for large object manipulation that uses human-object interaction demonstrations to train robot control policies. RobotMover formulates the manipulation problem as imitation learning using a simplified spatial representation called the Interaction Chain, which captures essential interaction dynamics in a way that generalizes across different robot bodies. We incorporate this Interaction Chain into a reward function and train policies in simulation using domain randomization to enable zero-shot transfer to real-world robots. The resulting policies allow a Spot robot to manipulate various large objects, including chairs, tables, and standing lamps. Through extensive experiments in both simulation and the real world, we show that RobotMover achieves strong performance in terms of capability, robustness, and controllability, outperforming both learned and teleoperation baselines. The system also supports practical applications by combining learned policies with simple planning modules to perform long-horizon object transport and rearrangement tasks.