CORE4D: A 4D Human-Object-Human Interaction Dataset for Collaborative Object REarrangement

TL;DR

CORE4D is a large-scale 4D dataset capturing human-human-object interactions for collaborative object rearrangement, enabling advancements in VR/AR and human-robot collaboration through diverse motion sequences and benchmarking tasks.

Contribution

We introduce CORE4D, a novel dataset with an innovative collaboration retargeting strategy, significantly expanding data diversity for human-object interaction research.

Findings

Retargeting strategy effectively augments motion data.

CORE4D poses new challenges for existing interaction models.

Benchmark results highlight the dataset's utility for motion forecasting and synthesis.

Abstract

Understanding how humans cooperatively rearrange household objects is critical for VR/AR and human-robot interaction. However, in-depth studies on modeling these behaviors are under-researched due to the lack of relevant datasets. We fill this gap by presenting CORE4D, a novel large-scale 4D human-object-human interaction dataset focusing on collaborative object rearrangement, which encompasses diverse compositions of various object geometries, collaboration modes, and 3D scenes. With 1K human-object-human motion sequences captured in the real world, we enrich CORE4D by contributing an iterative collaboration retargeting strategy to augment motions to a variety of novel objects. Leveraging this approach, CORE4D comprises a total of 11K collaboration sequences spanning 3K real and virtual object shapes. Benefiting from extensive motion patterns provided by CORE4D, we benchmark two tasks aiming at generating human-object interaction: human-object motion forecasting and interaction synthesis. Extensive experiments demonstrate the effectiveness of our collaboration retargeting strategy and indicate that CORE4D has posed new challenges to existing human-object interaction generation methodologies.