Domain adaptive pose estimation via multi-level alignment

TL;DR

This paper introduces a multi-level domain adaptation method for pose estimation that aligns images, features, and poses across domains, significantly improving accuracy in human and animal pose estimation tasks.

Contribution

The paper proposes a novel multi-level alignment approach combining image style transfer, adversarial feature training, and self-supervised pose learning for domain adaptation in pose estimation.

Findings

Outperforms previous methods in human pose estimation by up to 2.4%.

Achieves up to 3.1% improvement in animal pose estimation for dogs.

Demonstrates effective multi-level alignment reduces domain gap significantly.

Abstract

Domain adaptive pose estimation aims to enable deep models trained on source domain (synthesized) datasets produce similar results on the target domain (real-world) datasets. The existing methods have made significant progress by conducting image-level or feature-level alignment. However, only aligning at a single level is not sufficient to fully bridge the domain gap and achieve excellent domain adaptive results. In this paper, we propose a multi-level domain adaptation aproach, which aligns different domains at the image, feature, and pose levels. Specifically, we first utilize image style transer to ensure that images from the source and target domains have a similar distribution. Subsequently, at the feature level, we employ adversarial training to make the features from the source and target domains preserve domain-invariant characeristics as much as possible. Finally, at the pose level, a self-supervised approach is utilized to enable the model to learn diverse knowledge, implicitly addressing the domain gap. Experimental results demonstrate that significant imrovement can be achieved by the proposed multi-level alignment method in pose estimation, which outperforms previous state-of-the-art in human pose by up to 2.4% and animal pose estimation by up to 3.1% for dogs and 1.4% for sheep.