Physics-aware Hand-object Interaction Denoising

TL;DR

This paper presents a physically-aware de-noising method for hand-object interaction sequences that improves physical plausibility and pose accuracy by incorporating learned loss terms for grasp credibility and manipulation feasibility.

Contribution

It introduces a novel de-noising network with learned loss terms explicitly modeling physical plausibility aspects, enhancing existing hand tracking results.

Findings

Significantly improves physical plausibility of hand-object interactions.

Outperforms state-of-the-art de-noising methods in accuracy.

Enhances both qualitative and quantitative tracking results.

Abstract

The credibility and practicality of a reconstructed hand-object interaction sequence depend largely on its physical plausibility. However, due to high occlusions during hand-object interaction, physical plausibility remains a challenging criterion for purely vision-based tracking methods. To address this issue and enhance the results of existing hand trackers, this paper proposes a novel physically-aware hand motion de-noising method. Specifically, we introduce two learned loss terms that explicitly capture two crucial aspects of physical plausibility: grasp credibility and manipulation feasibility. These terms are used to train a physically-aware de-noising network. Qualitative and quantitative experiments demonstrate that our approach significantly improves both fine-grained physical plausibility and overall pose accuracy, surpassing current state-of-the-art de-noising methods.