Human Motion Estimation with Everyday Wearables

TL;DR

This paper introduces EveryWear, a practical human motion capture system using everyday wearables and a new real-world dataset, enabling accurate motion estimation without cumbersome calibration.

Contribution

The paper presents a novel multimodal framework and a real-world dataset for wearable-based human motion estimation, eliminating the need for calibration and synthetic data.

Findings

Outperforms baseline models in real-world scenarios

Effectively integrates visual and inertial data

Eliminates the sim-to-real gap in motion estimation

Abstract

While on-body device-based human motion estimation is crucial for applications such as XR interaction, existing methods often suffer from poor wearability, expensive hardware, and cumbersome calibration, which hinder their adoption in daily life. To address these challenges, we present EveryWear, a lightweight and practical human motion capture approach based entirely on everyday wearables: a smartphone, smartwatch, earbuds, and smart glasses equipped with one forward-facing and two downward-facing cameras, requiring no explicit calibration before use. We introduce Ego-Elec, a 9-hour real-world dataset covering 56 daily activities across 17 diverse indoor and outdoor environments, with ground-truth 3D annotations provided by the motion capture (MoCap), to facilitate robust research and benchmarking in this direction. Our approach employs a multimodal teacher-student framework that integrates visual cues from egocentric cameras with inertial signals from consumer devices. By training directly on real-world data rather than synthetic data, our model effectively eliminates the sim-to-real gap that constrains prior work. Experiments demonstrate that our method outperforms baseline models, validating its effectiveness for practical full-body motion estimation.