AdaptPose: Cross-Dataset Adaptation for 3D Human Pose Estimation by Learnable Motion Generation

TL;DR

AdaptPose is an end-to-end framework that improves cross-dataset generalization of 3D human pose estimation by generating synthetic motions and adapting to target dataset characteristics without requiring 3D labels.

Contribution

It introduces a novel adversarial training scheme for synthetic motion generation that enhances model adaptation to new datasets without 3D annotations.

Findings

Outperforms previous methods by 14% in cross-dataset evaluations.

Surpasses semi-supervised methods with partial 3D annotations by 16%.

Effectively adapts to diverse datasets like Human3.6M and 3DPW.

Abstract

This paper addresses the problem of cross-dataset generalization of 3D human pose estimation models. Testing a pre-trained 3D pose estimator on a new dataset results in a major performance drop. Previous methods have mainly addressed this problem by improving the diversity of the training data. We argue that diversity alone is not sufficient and that the characteristics of the training data need to be adapted to those of the new dataset such as camera viewpoint, position, human actions, and body size. To this end, we propose AdaptPose, an end-to-end framework that generates synthetic 3D human motions from a source dataset and uses them to fine-tune a 3D pose estimator. AdaptPose follows an adversarial training scheme. From a source 3D pose the generator generates a sequence of 3D poses and a camera orientation that is used to project the generated poses to a novel view. Without any 3D labels or camera information AdaptPose successfully learns to create synthetic 3D poses from the target dataset while only being trained on 2D poses. In experiments on the Human3.6M, MPI-INF-3DHP, 3DPW, and Ski-Pose datasets our method outperforms previous work in cross-dataset evaluations by 14% and previous semi-supervised learning methods that use partial 3D annotations by 16%.