Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Video

TL;DR

This paper introduces TCMR, a system that improves temporal consistency and accuracy in 3D human pose and shape estimation from videos by leveraging past and future frame information, surpassing previous methods.

Contribution

The paper proposes a novel temporally consistent mesh recovery system that effectively utilizes temporal information beyond static frame features, improving over existing video-based methods.

Findings

Outperforms previous methods in temporal consistency

Achieves better per-frame 3D pose and shape accuracy

Demonstrates effectiveness through quantitative results

Abstract

Despite the recent success of single image-based 3D human pose and shape estimation methods, recovering temporally consistent and smooth 3D human motion from a video is still challenging. Several video-based methods have been proposed; however, they fail to resolve the single image-based methods' temporal inconsistency issue due to a strong dependency on a static feature of the current frame. In this regard, we present a temporally consistent mesh recovery system (TCMR). It effectively focuses on the past and future frames' temporal information without being dominated by the current static feature. Our TCMR significantly outperforms previous video-based methods in temporal consistency with better per-frame 3D pose and shape accuracy. We also release the codes. For the demo video, see https://youtu.be/WB3nTnSQDII. For the codes, see https://github.com/hongsukchoi/TCMR_RELEASE.