DynaHOI: Benchmarking Hand-Object Interaction for Dynamic Target

TL;DR

This paper introduces DynaHOI-Gym and DynaHOI-10M, a comprehensive benchmark and platform for evaluating hand-object interaction in dynamic scenarios with moving targets, addressing a significant gap in existing static-focused benchmarks.

Contribution

We present a new benchmark and platform for dynamic hand-object interaction, including a large-scale dataset and a baseline method with improved success rate.

Findings

DynaHOI-10M contains 10 million frames and 180,000 trajectories.

The ObAct baseline improves location success rate by 8.1%.

DynaHOI-Gym enables evaluation of dynamic HOI with parameterized motions.

Abstract

Most existing hand motion generation benchmarks for hand-object interaction (HOI) focus on static objects, leaving dynamic scenarios with moving targets and time-critical coordination largely untested. To address this gap, we introduce the DynaHOI-Gym, a unified online closed-loop platform with parameterized motion generators and rollout-based metrics for dynamic capture evaluation. Built on DynaHOI-Gym, we release DynaHOI-10M, a large-scale benchmark with 10M frames and 180K hand capture trajectories, whose target motions are organized into 8 major categories and 22 fine-grained subcategories. We also provide a simple observe-before-act baseline (ObAct) that integrates short-term observations with the current frame via spatiotemporal attention to predict actions, achieving an 8.1% improvement in location success rate.