Learning Interaction-aware 3D Gaussian Splatting for One-shot Hand Avatars

TL;DR

This paper introduces a novel two-stage 3D Gaussian Splatting framework for creating animatable, interaction-aware hand avatars from a single image, effectively handling hand variations and interactions.

Contribution

The work presents a new interaction-aware Gaussian Splatting method that disentangles hand representations and incorporates attention and refinement modules for improved rendering quality.

Findings

Significant improvement in image quality over state-of-the-art methods.

Effective handling of hand variations and interactions.

Validated on large-scale InterHand2.6M dataset.

Abstract

In this paper, we propose to create animatable avatars for interacting hands with 3D Gaussian Splatting (GS) and single-image inputs. Existing GS-based methods designed for single subjects often yield unsatisfactory results due to limited input views, various hand poses, and occlusions. To address these challenges, we introduce a novel two-stage interaction-aware GS framework that exploits cross-subject hand priors and refines 3D Gaussians in interacting areas. Particularly, to handle hand variations, we disentangle the 3D presentation of hands into optimization-based identity maps and learning-based latent geometric features and neural texture maps. Learning-based features are captured by trained networks to provide reliable priors for poses, shapes, and textures, while optimization-based identity maps enable efficient one-shot fitting of out-of-distribution hands. Furthermore, we devise an interaction-aware attention module and a self-adaptive Gaussian refinement module. These modules enhance image rendering quality in areas with intra- and inter-hand interactions, overcoming the limitations of existing GS-based methods. Our proposed method is validated via extensive experiments on the large-scale InterHand2.6M dataset, and it significantly improves the state-of-the-art performance in image quality. Project Page: \url{https://github.com/XuanHuang0/GuassianHand}.