Exploiting Aggregation and Segregation of Representations for Domain Adaptive Human Pose Estimation

TL;DR

This paper proposes a novel domain adaptation framework for human pose estimation that leverages both aggregation of domain-invariant features and segregation of domain-specific features, improving performance across diverse datasets.

Contribution

It introduces a new approach that disentangles representations into domain-invariant and domain-specific parts, enhancing domain adaptation in human pose estimation.

Findings

Achieves state-of-the-art results on multiple benchmarks.

Effectively disentangles domain-invariant and domain-specific features.

Improves alignment and segregation in feature representations.

Abstract

Human pose estimation (HPE) has received increasing attention recently due to its wide application in motion analysis, virtual reality, healthcare, etc. However, it suffers from the lack of labeled diverse real-world datasets due to the time- and labor-intensive annotation. To cope with the label deficiency issue, one common solution is to train the HPE models with easily available synthetic datasets (source) and apply them to real-world data (target) through domain adaptation (DA). Unfortunately, prevailing domain adaptation techniques within the HPE domain remain predominantly fixated on effecting alignment and aggregation between source and target features, often sidestepping the crucial task of excluding domain-specific representations. To rectify this, we introduce a novel framework that capitalizes on both representation aggregation and segregation for domain adaptive human pose estimation. Within this framework, we address the network architecture aspect by disentangling representations into distinct domain-invariant and domain-specific components, facilitating aggregation of domain-invariant features while simultaneously segregating domain-specific ones. Moreover, we tackle the discrepancy measurement facet by delving into various keypoint relationships and applying separate aggregation or segregation mechanisms to enhance alignment. Extensive experiments on various benchmarks, e.g., Human3.6M, LSP, H3D, and FreiHand, show that our method consistently achieves state-of-the-art performance. The project is available at \url{https://github.com/davidpengucf/EPIC}.