Stochastic Scene-Aware Motion Prediction

TL;DR

This paper introduces SAMP, a data-driven stochastic method for realistic, scene-aware human motion synthesis that captures diverse styles and adapts to various indoor environments, improving over existing approaches.

Contribution

The paper presents a novel stochastic motion prediction model that generalizes to different objects and scenes, enabling realistic virtual human interactions in cluttered indoor environments.

Findings

SAMP outperforms existing methods in complex indoor scenes.

Collected diverse MoCap data for multiple human actions.

Demonstrates realistic and varied human-scene interactions.

Abstract

A long-standing goal in computer vision is to capture, model, and realistically synthesize human behavior. Specifically, by learning from data, our goal is to enable virtual humans to navigate within cluttered indoor scenes and naturally interact with objects. Such embodied behavior has applications in virtual reality, computer games, and robotics, while synthesized behavior can be used as a source of training data. This is challenging because real human motion is diverse and adapts to the scene. For example, a person can sit or lie on a sofa in many places and with varying styles. It is necessary to model this diversity when synthesizing virtual humans that realistically perform human-scene interactions. We present a novel data-driven, stochastic motion synthesis method that models different styles of performing a given action with a target object. Our method, called SAMP, for Scene-Aware Motion Prediction, generalizes to target objects of various geometries while enabling the character to navigate in cluttered scenes. To train our method, we collected MoCap data covering various sitting, lying down, walking, and running styles. We demonstrate our method on complex indoor scenes and achieve superior performance compared to existing solutions. Our code and data are available for research at https://samp.is.tue.mpg.de.