M3DHMR: Monocular 3D Hand Mesh Recovery

TL;DR

M3DHMR is a novel pipeline for monocular 3D hand mesh recovery that directly estimates mesh vertices using a dynamic spiral convolution approach, improving accuracy and efficiency over existing methods.

Contribution

The paper introduces a new direct vertex prediction pipeline with a spiral decoder and ROI refinement, advancing monocular 3D hand mesh recovery techniques.

Findings

Outperforms state-of-the-art methods on FreiHAND dataset

Achieves real-time performance with high accuracy

Effectively handles self-occlusion and 2D-3D ambiguity

Abstract

Monocular 3D hand mesh recovery is challenging due to high degrees of freedom of hands, 2D-to-3D ambiguity and self-occlusion. Most existing methods are either inefficient or less straightforward for predicting the position of 3D mesh vertices. Thus, we propose a new pipeline called Monocular 3D Hand Mesh Recovery (M3DHMR) to directly estimate the positions of hand mesh vertices. M3DHMR provides 2D cues for 3D tasks from a single image and uses a new spiral decoder consist of several Dynamic Spiral Convolution (DSC) Layers and a Region of Interest (ROI) Layer. On the one hand, DSC Layers adaptively adjust the weights based on the vertex positions and extract the vertex features in both spatial and channel dimensions. On the other hand, ROI Layer utilizes the physical information and refines mesh vertices in each predefined hand region separately. Extensive experiments on popular dataset FreiHAND demonstrate that M3DHMR significantly outperforms state-of-the-art real-time methods.