PACER+: On-Demand Pedestrian Animation Controller in Driving Scenarios

TL;DR

PACER+ introduces a novel pedestrian animation framework that combines motion tracking with trajectory following, enabling diverse, realistic, and controllable pedestrian behaviors for driving simulations, surpassing previous limitations.

Contribution

The paper presents a new framework that integrates motion tracking and trajectory following, allowing for diverse and controllable pedestrian animations in driving scenarios.

Findings

Enhanced diversity of pedestrian motions in simulations.

Improved controllability including language-based control.

Greater realism and adaptability in pedestrian behavior modeling.

Abstract

We address the challenge of content diversity and controllability in pedestrian simulation for driving scenarios. Recent pedestrian animation frameworks have a significant limitation wherein they primarily focus on either following trajectory [46] or the content of the reference video [57], consequently overlooking the potential diversity of human motion within such scenarios. This limitation restricts the ability to generate pedestrian behaviors that exhibit a wider range of variations and realistic motions and therefore restricts its usage to provide rich motion content for other components in the driving simulation system, e.g., suddenly changed motion to which the autonomous vehicle should respond. In our approach, we strive to surpass the limitation by showcasing diverse human motions obtained from various sources, such as generated human motions, in addition to following the given trajectory. The fundamental contribution of our framework lies in combining the motion tracking task with trajectory following, which enables the tracking of specific motion parts (e.g., upper body) while simultaneously following the given trajectory by a single policy. This way, we significantly enhance both the diversity of simulated human motion within the given scenario and the controllability of the content, including language-based control. Our framework facilitates the generation of a wide range of human motions, contributing to greater realism and adaptability in pedestrian simulations for driving scenarios. More information is on our project page https://wangjingbo1219.github.io/papers/CVPR2024_PACER_PLUS/PACERPLUSPage.html .