PoSynDA: Multi-Hypothesis Pose Synthesis Domain Adaptation for Robust 3D Human Pose Estimation

TL;DR

PoSynDA introduces a diffusion-inspired, multi-hypothesis domain adaptation framework that enhances 3D human pose estimation across unseen datasets by simulating target domain pose distributions and aligning diverse hypotheses.

Contribution

It proposes a novel multi-hypothesis domain adaptation method using diffusion-inspired structures and target-specific augmentation for robust 3D pose estimation in new domains.

Findings

Achieves performance comparable to target-trained models on benchmarks.

Effectively bridges the domain gap in 3D human pose estimation.

Demonstrates robustness across multiple datasets.

Abstract

Existing 3D human pose estimators face challenges in adapting to new datasets due to the lack of 2D-3D pose pairs in training sets. To overcome this issue, we propose \textit{Multi-Hypothesis \textbf{P}ose \textbf{Syn}thesis \textbf{D}omain \textbf{A}daptation} (\textbf{PoSynDA}) framework to bridge this data disparity gap in target domain. Typically, PoSynDA uses a diffusion-inspired structure to simulate 3D pose distribution in the target domain. By incorporating a multi-hypothesis network, PoSynDA generates diverse pose hypotheses and aligns them with the target domain. To do this, it first utilizes target-specific source augmentation to obtain the target domain distribution data from the source domain by decoupling the scale and position parameters. The process is then further refined through the teacher-student paradigm and low-rank adaptation. With extensive comparison of benchmarks such as Human3.6M and MPI-INF-3DHP, PoSynDA demonstrates competitive performance, even comparable to the target-trained MixSTE model\cite{zhang2022mixste}. This work paves the way for the practical application of 3D human pose estimation in unseen domains. The code is available at https://github.com/hbing-l/PoSynDA.