MaskedManipulator: Versatile Whole-Body Manipulation

TL;DR

MaskedManipulator is a new framework for generating versatile, goal-directed full-body human motions for object manipulation, combining large-scale motion data with user-defined high-level objectives.

Contribution

It introduces a two-stage learning approach that distills a generative control policy from a tracking controller trained on extensive motion capture data.

Findings

Enables complex interaction behaviors with intuitive user control.

Expands interactive animation capabilities beyond task-specific methods.

Produces goal-directed, versatile manipulation motions.

Abstract

We tackle the challenges of synthesizing versatile, physically simulated human motions for full-body object manipulation. Unlike prior methods that are focused on detailed motion tracking, trajectory following, or teleoperation, our framework enables users to specify versatile high-level objectives such as target object poses or body poses. To achieve this, we introduce MaskedManipulator, a generative control policy distilled from a tracking controller trained on large-scale human motion capture data. This two-stage learning process allows the system to perform complex interaction behaviors, while providing intuitive user control over both character and object motions. MaskedManipulator produces goal-directed manipulation behaviors that expand the scope of interactive animation systems beyond task-specific solutions.