HSC4D: Human-centered 4D Scene Capture in Large-scale Indoor-outdoor Space Using Wearable IMUs and LiDAR

TL;DR

HSC4D introduces a novel method for large-scale, real-time 4D scene capture using only wearable IMUs and LiDAR, enabling accurate modeling of dynamic indoor-outdoor environments without external devices or pre-built maps.

Contribution

The paper presents a new sensor fusion approach combining IMUs and LiDAR for long-term, large-scale 4D scene capture without external constraints or pre-mapped environments.

Findings

Effective long-term capture with joint optimization of sensors.

Realistic modeling of human-environment interactions.

Generalizes well across diverse large-scale scenes.

Abstract

We propose Human-centered 4D Scene Capture (HSC4D) to accurately and efficiently create a dynamic digital world, containing large-scale indoor-outdoor scenes, diverse human motions, and rich interactions between humans and environments. Using only body-mounted IMUs and LiDAR, HSC4D is space-free without any external devices' constraints and map-free without pre-built maps. Considering that IMUs can capture human poses but always drift for long-period use, while LiDAR is stable for global localization but rough for local positions and orientations, HSC4D makes both sensors complement each other by a joint optimization and achieves promising results for long-term capture. Relationships between humans and environments are also explored to make their interaction more realistic. To facilitate many down-stream tasks, like AR, VR, robots, autonomous driving, etc., we propose a dataset containing three large scenes (1k-5k $m^2$) with accurate dynamic human motions and locations. Diverse scenarios (climbing gym, multi-story building, slope, etc.) and challenging human activities (exercising, walking up/down stairs, climbing, etc.) demonstrate the effectiveness and the generalization ability of HSC4D. The dataset and code are available at http://www.lidarhumanmotion.net/hsc4d/.