Toward Efficient Generalization in 3D Human Pose Estimation via a Canonical Domain Approach

TL;DR

This paper introduces a canonical domain approach for 3D human pose estimation that reduces domain gap issues, enabling models to generalize better across datasets without extra fine-tuning.

Contribution

The paper proposes a novel canonicalization process that maps source and target domains into a unified domain, improving cross-dataset generalization in 3D human pose estimation.

Findings

Significant improvement in cross-dataset generalization performance

Method reduces need for fine-tuning on new datasets

Effective across various lifting networks and datasets

Abstract

Recent advancements in deep learning methods have significantly improved the performance of 3D Human Pose Estimation (HPE). However, performance degradation caused by domain gaps between source and target domains remains a major challenge to generalization, necessitating extensive data augmentation and/or fine-tuning for each specific target domain. To address this issue more efficiently, we propose a novel canonical domain approach that maps both the source and target domains into a unified canonical domain, alleviating the need for additional fine-tuning in the target domain. To construct the canonical domain, we introduce a canonicalization process to generate a novel canonical 2D-3D pose mapping that ensures 2D-3D pose consistency and simplifies 2D-3D pose patterns, enabling more efficient training of lifting networks. The canonicalization of both domains is achieved through the following steps: (1) in the source domain, the lifting network is trained within the canonical domain; (2) in the target domain, input 2D poses are canonicalized prior to inference by leveraging the properties of perspective projection and known camera intrinsics. Consequently, the trained network can be directly applied to the target domain without requiring additional fine-tuning. Experiments conducted with various lifting networks and publicly available datasets (e.g., Human3.6M, Fit3D, MPI-INF-3DHP) demonstrate that the proposed method substantially improves generalization capability across datasets while using the same data volume.