UPTor: Unified 3D Human Pose Dynamics and Trajectory Prediction for Human-Robot Interaction

TL;DR

UPTor presents a unified, real-time model that accurately predicts 3D human pose dynamics and trajectories, enhancing human-robot interaction and navigation by merging pose and motion forecasting.

Contribution

It introduces a novel, integrated approach combining pose and trajectory prediction with a new dataset, enabling real-time human-aware navigation in robotics.

Findings

Outperforms prior models in accuracy across datasets.

Operates in real-time suitable for robotic applications.

Provides a new dataset focused on navigational activities.

Abstract

We introduce a unified approach to forecast the dynamics of human keypoints along with the motion trajectory based on a short sequence of input poses. While many studies address either full-body pose prediction or motion trajectory prediction, only a few attempt to merge them. We propose a motion transformation technique to simultaneously predict full-body pose and trajectory key-points in a global coordinate frame. We utilize an off-the-shelf 3D human pose estimation module, a graph attention network to encode the skeleton structure, and a compact, non-autoregressive transformer suitable for real-time motion prediction for human-robot interaction and human-aware navigation. We introduce a human navigation dataset ``DARKO'' with specific focus on navigational activities that are relevant for human-aware mobile robot navigation. We perform extensive evaluation on Human3.6M, CMU-Mocap, and our DARKO dataset. In comparison to prior work, we show that our approach is compact, real-time, and accurate in predicting human navigation motion across all datasets. Result animations, our dataset, and code will be available at https://nisarganc.github.io/UPTor-page/