Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusion

TL;DR

This paper presents a controllable pedestrian animation system combining guided diffusion for trajectory generation with a physics-based humanoid controller, enabling realistic, goal-oriented crowd simulations adaptable to various environments.

Contribution

It introduces a novel guided diffusion approach for controllable pedestrian trajectories integrated with a physics-based controller for full-body animations, enhancing realism and user control.

Findings

Achieves test-time controllability of trajectories via guided diffusion.

Successfully integrates diffusion with physics-based humanoid control.

Demonstrates realistic crowd animations in diverse terrains.

Abstract

We introduce a method for generating realistic pedestrian trajectories and full-body animations that can be controlled to meet user-defined goals. We draw on recent advances in guided diffusion modeling to achieve test-time controllability of trajectories, which is normally only associated with rule-based systems. Our guided diffusion model allows users to constrain trajectories through target waypoints, speed, and specified social groups while accounting for the surrounding environment context. This trajectory diffusion model is integrated with a novel physics-based humanoid controller to form a closed-loop, full-body pedestrian animation system capable of placing large crowds in a simulated environment with varying terrains. We further propose utilizing the value function learned during RL training of the animation controller to guide diffusion to produce trajectories better suited for particular scenarios such as collision avoidance and traversing uneven terrain. Video results are available on the project page at https://nv-tlabs.github.io/trace-pace .