Aligning Silhouette Topology for Self-Adaptive 3D Human Pose Recovery

TL;DR

This paper introduces a silhouette-based domain adaptation method for 3D human pose estimation that leverages topology-aware loss functions to improve model robustness across diverse real-world scenarios without requiring extensive annotations.

Contribution

It proposes a novel silhouette topology alignment framework using spatial transformations and a Chamfer-inspired loss for effective self-adaptation in unlabeled target domains.

Findings

Outperforms prior methods in diverse in-the-wild datasets

Effective in low-resolution and adversarially perturbed images

Achieves robust 3D pose estimation without auxiliary cues

Abstract

Articulation-centric 2D/3D pose supervision forms the core training objective in most existing 3D human pose estimation techniques. Except for synthetic source environments, acquiring such rich supervision for each real target domain at deployment is highly inconvenient. However, we realize that standard foreground silhouette estimation techniques (on static camera feeds) remain unaffected by domain-shifts. Motivated by this, we propose a novel target adaptation framework that relies only on silhouette supervision to adapt a source-trained model-based regressor. However, in the absence of any auxiliary cue (multi-view, depth, or 2D pose), an isolated silhouette loss fails to provide a reliable pose-specific gradient and requires to be employed in tandem with a topology-centric loss. To this end, we develop a series of convolution-friendly spatial transformations in order to disentangle a topological-skeleton representation from the raw silhouette. Such a design paves the way to devise a Chamfer-inspired spatial topological-alignment loss via distance field computation, while effectively avoiding any gradient hindering spatial-to-pointset mapping. Experimental results demonstrate our superiority against prior-arts in self-adapting a source trained model to diverse unlabeled target domains, such as a) in-the-wild datasets, b) low-resolution image domains, and c) adversarially perturbed image domains (via UAP).