GRACE: Estimating Geometry-level 3D Human-Scene Contact from 2D Images

TL;DR

GRACE introduces a novel 3D human-scene contact estimation method that effectively integrates geometric structures with 2D image semantics, achieving state-of-the-art accuracy and strong generalization across diverse human geometries.

Contribution

The paper presents GRACE, a point cloud-based framework that models geometry-level contact estimation, overcoming limitations of parametric models and enhancing generalization.

Findings

Achieves state-of-the-art contact estimation accuracy.

Demonstrates robust generalization to unstructured human point clouds.

Outperforms existing methods on multiple benchmarks.

Abstract

Estimating the geometry level of human-scene contact aims to ground specific contact surface points at 3D human geometries, which provides a spatial prior and bridges the interaction between human and scene, supporting applications such as human behavior analysis, embodied AI, and AR/VR. To complete the task, existing approaches predominantly rely on parametric human models (e.g., SMPL), which establish correspondences between images and contact regions through fixed SMPL vertex sequences. This actually completes the mapping from image features to an ordered sequence. However, this approach lacks consideration of geometry, limiting its generalizability in distinct human geometries. In this paper, we introduce GRACE (Geometry-level Reasoning for 3D Human-scene Contact Estimation), a new paradigm for 3D human contact estimation. GRACE incorporates a point cloud encoder-decoder architecture along with a hierarchical feature extraction and fusion module, enabling the effective integration of 3D human geometric structures with 2D interaction semantics derived from images. Guided by visual cues, GRACE establishes an implicit mapping from geometric features to the vertex space of the 3D human mesh, thereby achieving accurate modeling of contact regions. This design ensures high prediction accuracy and endows the framework with strong generalization capability across diverse human geometries. Extensive experiments on multiple benchmark datasets demonstrate that GRACE achieves state-of-the-art performance in contact estimation, with additional results further validating its robust generalization to unstructured human point clouds.