A Skeleton-Driven Neural Occupancy Representation for Articulated Hands

TL;DR

This paper introduces HALO, a neural occupancy representation for articulated hands that uses 3D keypoints for improved accuracy and can be trained end-to-end for better hand-object grasp synthesis.

Contribution

HALO is a novel hand representation that combines 3D keypoints with neural implicit surfaces, enabling end-to-end training and improved hand modeling.

Findings

HALO improves the physical plausibility of generated hand grasps.

HALO outperforms traditional models in hand surface accuracy.

End-to-end training enhances hand-object interaction quality.

Abstract

We present Hand ArticuLated Occupancy (HALO), a novel representation of articulated hands that bridges the advantages of 3D keypoints and neural implicit surfaces and can be used in end-to-end trainable architectures. Unlike existing statistical parametric hand models (e.g.~MANO), HALO directly leverages 3D joint skeleton as input and produces a neural occupancy volume representing the posed hand surface. The key benefits of HALO are (1) it is driven by 3D key points, which have benefits in terms of accuracy and are easier to learn for neural networks than the latent hand-model parameters; (2) it provides a differentiable volumetric occupancy representation of the posed hand; (3) it can be trained end-to-end, allowing the formulation of losses on the hand surface that benefit the learning of 3D keypoints. We demonstrate the applicability of HALO to the task of conditional generation of hands that grasp 3D objects. The differentiable nature of HALO is shown to improve the quality of the synthesized hands both in terms of physical plausibility and user preference.