HaWoR: World-Space Hand Motion Reconstruction from Egocentric Videos
Jinglei Zhang, Jiankang Deng, Chao Ma, Rolandos Alexandros Potamias

TL;DR
HaWoR introduces a novel approach for reconstructing 3D hand motion in world coordinates from egocentric videos, utilizing a decoupled framework with an adaptive SLAM and motion infiller network, achieving state-of-the-art results.
Contribution
The paper presents a new method for world-space hand motion reconstruction from egocentric videos, including a robust SLAM framework and a motion infiller network, addressing limitations of existing single-image methods.
Findings
Achieves state-of-the-art performance on hand motion reconstruction.
Demonstrates robust camera trajectory estimation in egocentric videos.
Effectively completes missing hand motion frames with the motion infiller network.
Abstract
Despite the advent in 3D hand pose estimation, current methods predominantly focus on single-image 3D hand reconstruction in the camera frame, overlooking the world-space motion of the hands. Such limitation prohibits their direct use in egocentric video settings, where hands and camera are continuously in motion. In this work, we propose HaWoR, a high-fidelity method for hand motion reconstruction in world coordinates from egocentric videos. We propose to decouple the task by reconstructing the hand motion in the camera space and estimating the camera trajectory in the world coordinate system. To achieve precise camera trajectory estimation, we propose an adaptive egocentric SLAM framework that addresses the shortcomings of traditional SLAM methods, providing robust performance under challenging camera dynamics. To ensure robust hand motion trajectories, even when the hands move out of…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHuman Pose and Action Recognition · Human Motion and Animation · Hand Gesture Recognition Systems
MethodsFocus
