Lifelong Domain Adaptive 3D Human Pose Estimation

TL;DR

This paper introduces the first lifelong domain adaptation framework for 3D human pose estimation, enabling models to adapt to new domains sequentially without forgetting previous ones, using a novel GAN-based approach.

Contribution

It proposes a pioneering lifelong domain adaptation setting for 3D HPE and develops a GAN framework with specialized pose generators to address domain shifts and catastrophic forgetting.

Findings

Outperforms existing methods on multiple domain adaptive 3D HPE datasets.

Effectively mitigates catastrophic forgetting in sequential domain adaptation.

Enhances 3D pose estimation accuracy across diverse in-the-wild scenarios.

Abstract

3D Human Pose Estimation (3D HPE) is vital in various applications, from person re-identification and action recognition to virtual reality. However, the reliance on annotated 3D data collected in controlled environments poses challenges for generalization to diverse in-the-wild scenarios. Existing domain adaptation (DA) paradigms like general DA and source-free DA for 3D HPE overlook the issues of non-stationary target pose datasets. To address these challenges, we propose a novel task named lifelong domain adaptive 3D HPE. To our knowledge, we are the first to introduce the lifelong domain adaptation to the 3D HPE task. In this lifelong DA setting, the pose estimator is pretrained on the source domain and subsequently adapted to distinct target domains. Moreover, during adaptation to the current target domain, the pose estimator cannot access the source and all the previous target domains. The lifelong DA for 3D HPE involves overcoming challenges in adapting to current domain poses and preserving knowledge from previous domains, particularly combating catastrophic forgetting. We present an innovative Generative Adversarial Network (GAN) framework, which incorporates 3D pose generators, a 2D pose discriminator, and a 3D pose estimator. This framework effectively mitigates domain shifts and aligns original and augmented poses. Moreover, we construct a novel 3D pose generator paradigm, integrating pose-aware, temporal-aware, and domain-aware knowledge to enhance the current domain's adaptation and alleviate catastrophic forgetting on previous domains. Our method demonstrates superior performance through extensive experiments on diverse domain adaptive 3D HPE datasets.