Physical Interaction: Reconstructing Hand-object Interactions with Physics

TL;DR

This paper introduces a physics-based reconstruction method for hand-object interactions from a single RGBD view, improving accuracy and plausibility by modeling contact forces and motion.

Contribution

It presents a novel force-based dynamic model and a confidence-based slide prevention scheme for more accurate and physically plausible hand-object interaction reconstruction.

Findings

Reconstructs physically plausible hand-object interactions in real-time.

Accurately estimates contact forces from a single RGBD sensor.

Outperforms previous methods in qualitative and quantitative evaluations.

Abstract

Single view-based reconstruction of hand-object interaction is challenging due to the severe observation missing caused by occlusions. This paper proposes a physics-based method to better solve the ambiguities in the reconstruction. It first proposes a force-based dynamic model of the in-hand object, which not only recovers the unobserved contacts but also solves for plausible contact forces. Next, a confidence-based slide prevention scheme is proposed, which combines both the kinematic confidences and the contact forces to jointly model static and sliding contact motion. Qualitative and quantitative experiments show that the proposed technique reconstructs both physically plausible and more accurate hand-object interaction and estimates plausible contact forces in real-time with a single RGBD sensor.