Human Motion Prediction via Test-domain-aware Adaptation with Easily-available Human Motions Estimated from Videos

TL;DR

This paper introduces a method to improve 3D human motion prediction by adapting models using estimated 3D poses derived from easily accessible videos, reducing reliance on costly motion capture data.

Contribution

It presents a novel pipeline to convert 2D video poses into 3D motions and uses these for domain-aware adaptation of HMP models, enhancing generalization.

Findings

Improved prediction accuracy on unseen motions.

Effective domain adaptation using estimated motions.

Qualitative improvements in motion realism.

Abstract

In 3D Human Motion Prediction (HMP), conventional methods train HMP models with expensive motion capture data. However, the data collection cost of such motion capture data limits the data diversity, which leads to poor generalizability to unseen motions or subjects. To address this issue, this paper proposes to enhance HMP with additional learning using estimated poses from easily available videos. The 2D poses estimated from the monocular videos are carefully transformed into motion capture-style 3D motions through our pipeline. By additional learning with the obtained motions, the HMP model is adapted to the test domain. The experimental results demonstrate the quantitative and qualitative impact of our method.