Mass Displacement Networks

TL;DR

Mass Displacement Networks (MDNs) integrate geometric post-processing into deep learning models for improved human pose estimation, enabling more precise and globally consistent keypoint localization.

Contribution

The paper introduces MDNs, a differentiable and probabilistically sound method for geometric evidence accumulation within neural networks, enhancing pose estimation accuracy.

Findings

Systematic improvements on MPII and COCO benchmarks.

Enhanced localization precision of body keypoints.

Better global consistency in pose estimation.

Abstract

Despite the large improvements in performance attained by using deep learning in computer vision, one can often further improve results with some additional post-processing that exploits the geometric nature of the underlying task. This commonly involves displacing the posterior distribution of a CNN in a way that makes it more appropriate for the task at hand, e.g. better aligned with local image features, or more compact. In this work we integrate this geometric post-processing within a deep architecture, introducing a differentiable and probabilistically sound counterpart to the common geometric voting technique used for evidence accumulation in vision. We refer to the resulting neural models as Mass Displacement Networks (MDNs), and apply them to human pose estimation in two distinct setups: (a) landmark localization, where we collapse a distribution to a point, allowing for precise localization of body keypoints and (b) communication across body parts, where we transfer evidence from one part to the other, allowing for a globally consistent pose estimate. We evaluate on large-scale pose estimation benchmarks, such as MPII Human Pose and COCO datasets, and report systematic improvements when compared to strong baselines.