Spatio-temporal Tendency Reasoning for Human Body Pose and Shape Estimation from Videos

TL;DR

This paper introduces a spatio-temporal tendency reasoning network that improves human pose and shape estimation from videos by effectively modeling temporal and spatial features, leading to more accurate and natural motion sequences.

Contribution

The paper proposes novel TTR and STE modules for hierarchical temporal and spatial feature reasoning, enhancing spatio-temporal feature extraction in human pose estimation.

Findings

Achieves competitive results on three large-scale datasets.

Effectively models temporal tendencies with hierarchical residual connections.

Enhances spatial features using frequency domain sensitivity.

Abstract

In this paper, we present a spatio-temporal tendency reasoning (STR) network for recovering human body pose and shape from videos. Previous approaches have focused on how to extend 3D human datasets and temporal-based learning to promote accuracy and temporal smoothing. Different from them, our STR aims to learn accurate and natural motion sequences in an unconstrained environment through temporal and spatial tendency and to fully excavate the spatio-temporal features of existing video data. To this end, our STR learns the representation of features in the temporal and spatial dimensions respectively, to concentrate on a more robust representation of spatio-temporal features. More specifically, for efficient temporal modeling, we first propose a temporal tendency reasoning (TTR) module. TTR constructs a time-dimensional hierarchical residual connection representation within a video sequence to effectively reason temporal sequences' tendencies and retain effective dissemination of human information. Meanwhile, for enhancing the spatial representation, we design a spatial tendency enhancing (STE) module to further learns to excite spatially time-frequency domain sensitive features in human motion information representations. Finally, we introduce integration strategies to integrate and refine the spatio-temporal feature representations. Extensive experimental findings on large-scale publically available datasets reveal that our STR remains competitive with the state-of-the-art on three datasets. Our code are available at https://github.com/Changboyang/STR.git.