TriPose: A Weakly-Supervised 3D Human Pose Estimation via Triangulation from Video

TL;DR

TriPose introduces a weakly-supervised approach for 3D human pose estimation from video that leverages multi-view triangulation and temporal information without needing 3D annotations or camera calibration.

Contribution

The paper proposes a novel weakly-supervised training scheme using multi-view triangulation and a recurrent lifting network, eliminating the need for 3D annotations and calibrated cameras.

Findings

Outperforms previous methods on Human3.6M and MPI-INF-3DHP datasets.

Does not require 3D annotations or camera calibration for training.

Effective in in-the-wild settings with only multi-view videos.

Abstract

Estimating 3D human poses from video is a challenging problem. The lack of 3D human pose annotations is a major obstacle for supervised training and for generalization to unseen datasets. In this work, we address this problem by proposing a weakly-supervised training scheme that does not require 3D annotations or calibrated cameras. The proposed method relies on temporal information and triangulation. Using 2D poses from multiple views as the input, we first estimate the relative camera orientations and then generate 3D poses via triangulation. The triangulation is only applied to the views with high 2D human joint confidence. The generated 3D poses are then used to train a recurrent lifting network (RLN) that estimates 3D poses from 2D poses. We further apply a multi-view re-projection loss to the estimated 3D poses and enforce the 3D poses estimated from multi-views to be consistent. Therefore, our method relaxes the constraints in practice, only multi-view videos are required for training, and is thus convenient for in-the-wild settings. At inference, RLN merely requires single-view videos. The proposed method outperforms previous works on two challenging datasets, Human3.6M and MPI-INF-3DHP. Codes and pretrained models will be publicly available.