Rotation-invariant Mixed Graphical Model Network for 2D Hand Pose Estimation

TL;DR

This paper introduces R-MGMN, a rotation-invariant neural network architecture for 2D hand pose estimation that outperforms existing methods by integrating graphical models and belief propagation.

Contribution

The paper presents a novel rotation-invariant network with a graphical model pool and belief propagation for improved 2D hand pose estimation.

Findings

Outperforms state-of-the-art algorithms on public datasets

Achieves higher accuracy in hand keypoint localization

Demonstrates robustness to hand rotations

Abstract

In this paper, we propose a new architecture named Rotation-invariant Mixed Graphical Model Network (R-MGMN) to solve the problem of 2D hand pose estimation from a monocular RGB image. By integrating a rotation net, the R-MGMN is invariant to rotations of the hand in the image. It also has a pool of graphical models, from which a combination of graphical models could be selected, conditioning on the input image. Belief propagation is performed on each graphical model separately, generating a set of marginal distributions, which are taken as the confidence maps of hand keypoint positions. Final confidence maps are obtained by aggregating these confidence maps together. We evaluate the R-MGMN on two public hand pose datasets. Experiment results show our model outperforms the state-of-the-art algorithm which is widely used in 2D hand pose estimation by a noticeable margin.