Shape Consistent 2D Keypoint Estimation under Domain Shift

TL;DR

This paper introduces a novel deep learning framework for 2D keypoint estimation that effectively handles domain shift by combining feature alignment, adversarial training, and self-supervision, leading to improved performance on benchmark datasets.

Contribution

The paper proposes a new domain adaptation method for 2D keypoint estimation that integrates feature alignment, adversarial loss, and geometric consistency, which was not previously combined in this way.

Findings

Outperforms state-of-the-art methods on three benchmarks

Effective domain adaptation for keypoint estimation

Combines multiple components for robust predictions

Abstract

Recent unsupervised domain adaptation methods based on deep architectures have shown remarkable performance not only in traditional classification tasks but also in more complex problems involving structured predictions (e.g. semantic segmentation, depth estimation). Following this trend, in this paper we present a novel deep adaptation framework for estimating keypoints under domain shift}, i.e. when the training (source) and the test (target) images significantly differ in terms of visual appearance. Our method seamlessly combines three different components: feature alignment, adversarial training and self-supervision. Specifically, our deep architecture leverages from domain-specific distribution alignment layers to perform target adaptation at the feature level. Furthermore, a novel loss is proposed which combines an adversarial term for ensuring aligned predictions in the output space and a geometric consistency term which guarantees coherent predictions between a target sample and its perturbed version. Our extensive experimental evaluation conducted on three publicly available benchmarks shows that our approach outperforms state-of-the-art domain adaptation methods in the 2D keypoint prediction task.