EventEgo3D++: 3D Human Motion Capture from a Head-Mounted Event Camera
Christen Millerdurai, Hiroyasu Akada, Jian Wang, Diogo, Luvizon, Alain Pagani, Didier Stricker, Christian Theobalt and, Vladislav Golyanik

TL;DR
EventEgo3D++ introduces a novel 3D human motion capture method using a monocular event camera on head-mounted devices, excelling in low light and fast movements, with real-time performance and comprehensive datasets.
Contribution
It is the first approach to leverage a monocular event camera with a fisheye lens for egocentric 3D human motion capture, including a new dataset and real-time capabilities.
Findings
Achieves superior 3D accuracy and robustness in challenging conditions
Supports real-time 3D pose updates at 140Hz
Demonstrates effectiveness on real and synthetic datasets
Abstract
Monocular egocentric 3D human motion capture remains a significant challenge, particularly under conditions of low lighting and fast movements, which are common in head-mounted device applications. Existing methods that rely on RGB cameras often fail under these conditions. To address these limitations, we introduce EventEgo3D++, the first approach that leverages a monocular event camera with a fisheye lens for 3D human motion capture. Event cameras excel in high-speed scenarios and varying illumination due to their high temporal resolution, providing reliable cues for accurate 3D human motion capture. EventEgo3D++ leverages the LNES representation of event streams to enable precise 3D reconstructions. We have also developed a mobile head-mounted device (HMD) prototype equipped with an event camera, capturing a comprehensive dataset that includes real event observations from both…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDigital Radiography and Breast Imaging · Functional Brain Connectivity Studies · Radiation Detection and Scintillator Technologies
