OmniControl: Control Any Joint at Any Time for Human Motion Generation

TL;DR

OmniControl introduces a unified diffusion-based model that enables flexible, joint-specific spatial control in human motion generation, improving realism and adherence to constraints over previous pelvis-only control methods.

Contribution

The paper presents a novel analytic spatial guidance method combined with realism guidance, allowing control over multiple joints at different times within a single model.

Findings

Significant improvement in pelvis control accuracy.

Effective incorporation of multi-joint spatial constraints.

Enhanced motion realism and coherence.

Abstract

We present a novel approach named OmniControl for incorporating flexible spatial control signals into a text-conditioned human motion generation model based on the diffusion process. Unlike previous methods that can only control the pelvis trajectory, OmniControl can incorporate flexible spatial control signals over different joints at different times with only one model. Specifically, we propose analytic spatial guidance that ensures the generated motion can tightly conform to the input control signals. At the same time, realism guidance is introduced to refine all the joints to generate more coherent motion. Both the spatial and realism guidance are essential and they are highly complementary for balancing control accuracy and motion realism. By combining them, OmniControl generates motions that are realistic, coherent, and consistent with the spatial constraints. Experiments on HumanML3D and KIT-ML datasets show that OmniControl not only achieves significant improvement over state-of-the-art methods on pelvis control but also shows promising results when incorporating the constraints over other joints.