GigaHands: A Massive Annotated Dataset of Bimanual Hand Activities

TL;DR

GigaHands is a large-scale, richly annotated dataset of bimanual hand activities, designed to advance AI and robotics by providing extensive motion data, object interactions, and text annotations.

Contribution

It introduces GigaHands, the first massive dataset with automatic 3D hand/object estimation, covering diverse activities and enabling broad applications in AI and robotics.

Findings

Contains 14,000 motion clips from 56 subjects and 417 objects.

Includes 84,000 text annotations paired with motion data.

Facilitates applications like action synthesis and hand motion captioning.

Abstract

Understanding bimanual human hand activities is a critical problem in AI and robotics. We cannot build large models of bimanual activities because existing datasets lack the scale, coverage of diverse hand activities, and detailed annotations. We introduce GigaHands, a massive annotated dataset capturing 34 hours of bimanual hand activities from 56 subjects and 417 objects, totaling 14k motion clips derived from 183 million frames paired with 84k text annotations. Our markerless capture setup and data acquisition protocol enable fully automatic 3D hand and object estimation while minimizing the effort required for text annotation. The scale and diversity of GigaHands enable broad applications, including text-driven action synthesis, hand motion captioning, and dynamic radiance field reconstruction. Our website are avaliable at https://ivl.cs.brown.edu/research/gigahands.html .