Skeleton-Graph: Long-Term 3D Motion Prediction From 2D Observations Using Deep Spatio-Temporal Graph CNNs

TL;DR

Skeleton-Graph is a deep spatio-temporal graph CNN model that accurately predicts long-term 3D human poses from 2D observations by modeling joint interactions, outperforming prior methods in accuracy and divergence metrics.

Contribution

The paper introduces Skeleton-Graph, a novel graph CNN approach that models joint interactions for long-term 3D pose prediction from 2D data, with improved accuracy and divergence measures.

Findings

Achieves at least 27% improvement in FDE on GTA-IM and PROX datasets.

Reduces divergence in long-term predictions by 88% and 93%.

Outperforms prior methods in both accuracy and stability of predictions.

Abstract

Several applications such as autonomous driving, augmented reality and virtual reality require a precise prediction of the 3D human pose. Recently, a new problem was introduced in the field to predict the 3D human poses from observed 2D poses. We propose Skeleton-Graph, a deep spatio-temporal graph CNN model that predicts the future 3D skeleton poses in a single pass from the 2D ones. Unlike prior works, Skeleton-Graph focuses on modeling the interaction between the skeleton joints by exploiting their spatial configuration. This is being achieved by formulating the problem as a graph structure while learning a suitable graph adjacency kernel. By the design, Skeleton-Graph predicts the future 3D poses without divergence in the long-term, unlike prior works. We also introduce a new metric that measures the divergence of predictions in the long term. Our results show an FDE improvement of at least 27% and an ADE of 4% on both the GTA-IM and PROX datasets respectively in comparison with prior works. Also, we are 88% and 93% less divergence on the long-term motion prediction in comparison with prior works on both GTA-IM and PROX datasets. Code is available at https://github.com/abduallahmohamed/Skeleton-Graph.git