Human Pose Estimation in Space and Time using 3D CNN

TL;DR

This paper presents a 3D CNN approach for estimating human 3D pose from monocular RGB videos, achieving state-of-the-art results by encoding temporal information as a third dimension in convolutional space.

Contribution

It introduces a novel 3D convolutional neural network architecture that effectively captures spatiotemporal features for 3D human pose estimation from monocular videos.

Findings

Achieved state-of-the-art performance on Human3.6M dataset.

Demonstrated effective encoding of temporal data as a 3D convolutional dimension.

Validated the approach's capability to regress 3D joint positions from RGB videos.

Abstract

This paper explores the capabilities of convolutional neural networks to deal with a task that is easily manageable for humans: perceiving 3D pose of a human body from varying angles. However, in our approach, we are restricted to using a monocular vision system. For this purpose, we apply a convolutional neural network approach on RGB videos and extend it to three dimensional convolutions. This is done via encoding the time dimension in videos as the 3\ts{rd} dimension in convolutional space, and directly regressing to human body joint positions in 3D coordinate space. This research shows the ability of such a network to achieve state-of-the-art performance on the selected Human3.6M dataset, thus demonstrating the possibility of successfully representing temporal data with an additional dimension in the convolutional operation.