DECA: Deep viewpoint-Equivariant human pose estimation using Capsule Autoencoders

TL;DR

DECA introduces a capsule autoencoder for human pose estimation that achieves viewpoint equivariance, enabling better generalization to unseen viewpoints and outperforming existing methods on depth and RGB images.

Contribution

The paper presents a novel capsule autoencoder with fast Variational Bayes routing that models joints as capsules, improving viewpoint generalization in human pose estimation.

Findings

Outperforms existing methods on depth images from unseen viewpoints

Achieves state-of-the-art results on RGB viewpoint transfer tasks

Reduces data dependency for training in human pose estimation

Abstract

Human Pose Estimation (HPE) aims at retrieving the 3D position of human joints from images or videos. We show that current 3D HPE methods suffer a lack of viewpoint equivariance, namely they tend to fail or perform poorly when dealing with viewpoints unseen at training time. Deep learning methods often rely on either scale-invariant, translation-invariant, or rotation-invariant operations, such as max-pooling. However, the adoption of such procedures does not necessarily improve viewpoint generalization, rather leading to more data-dependent methods. To tackle this issue, we propose a novel capsule autoencoder network with fast Variational Bayes capsule routing, named DECA. By modeling each joint as a capsule entity, combined with the routing algorithm, our approach can preserve the joints' hierarchical and geometrical structure in the feature space, independently from the viewpoint. By achieving viewpoint equivariance, we drastically reduce the network data dependency at training time, resulting in an improved ability to generalize for unseen viewpoints. In the experimental validation, we outperform other methods on depth images from both seen and unseen viewpoints, both top-view, and front-view. In the RGB domain, the same network gives state-of-the-art results on the challenging viewpoint transfer task, also establishing a new framework for top-view HPE. The code can be found at https://github.com/mmlab-cv/DECA.