Multi-view Image-based Hand Geometry Refinement using Differentiable Monte Carlo Ray Tracing

TL;DR

This paper introduces a novel hand geometry refinement method using differentiable Monte Carlo ray tracing to improve annotation quality in multi-view hand datasets, validated through synthetic and real data evaluations.

Contribution

It pioneers the use of differentiable ray tracing for hand dataset refinement, enhancing annotation accuracy beyond previous approximative methods.

Findings

Significant improvement in annotation quality demonstrated on InterHand2.6M dataset.

Differentiable ray tracing outperforms previous approximative techniques.

Visual evaluation confirms enhanced dataset quality in real data.

Abstract

The amount and quality of datasets and tools available in the research field of hand pose and shape estimation act as evidence to the significant progress that has been made.However, even the datasets of the highest quality, reported to date, have shortcomings in annotation. We propose a refinement approach, based on differentiable ray tracing,and demonstrate how a high-quality publicly available, multi-camera dataset of hands(InterHand2.6M) can become an even better dataset, with respect to annotation quality. Differentiable ray tracing has not been employed so far to relevant problems and is hereby shown to be superior to the approximative alternatives that have been employed in the past. To tackle the lack of reliable ground truth, as far as quantitative evaluation is concerned, we resort to realistic synthetic data, to show that the improvement we induce is indeed significant. The same becomes evident in real data through visual evaluation.