ReActor: Reinforcement Learning for Physics-Aware Motion Retargeting

TL;DR

ReActor introduces a physics-aware reinforcement learning framework that retargets human motion onto various robot morphologies, ensuring physically plausible motions for improved imitation learning.

Contribution

It presents a bilevel optimization approach that jointly adapts motions and trains policies without manual tuning, applicable to diverse robot forms including quadrupeds.

Findings

Successfully retargeted motions onto different robot morphologies in simulation.

Produced physically plausible motions that enhance imitation learning.

Validated on hardware with challenging retargeted motions.

Abstract

Retargeting human kinematic reference motion onto a robot's morphology remains a formidable challenge. Existing methods often produce physical inconsistencies, such as foot sliding, self-collisions, or dynamically infeasible motions, which hinder downstream imitation learning. We propose a bilevel optimization framework that jointly adapts reference motions to a robot's morphology while training a tracking policy using reinforcement learning. To make the optimization tractable, we derive an approximate gradient for the upper-level loss. Our framework requires only a sparse set of semantic rigid-body correspondences and eliminates the need for manual tuning by identifying optimal values for a parameterization expressive enough to preserve characteristic motion across different embodiments. Moreover, by integrating retargeting directly with physics simulation, we produce physically plausible motions that facilitate robust imitation learning. We validate our method in simulation and on hardware, demonstrating challenging motions for morphologies that differ significantly from a human, including retargeting onto a quadruped.