EgoForce: Robust Online Egocentric Motion Reconstruction via Diffusion Forcing

TL;DR

EgoForce is an online diffusion-based framework that reconstructs long-term full-body motion from noisy egocentric data, outperforming existing methods in real-time scenarios.

Contribution

It introduces a diffusion forcing approach with temporally asymmetric noise scheduling for robust, real-time egocentric motion reconstruction.

Findings

Outperforms existing online and offline methods in long-horizon motion reconstruction.

Provides stable and coherent full-body motion from noisy streaming egocentric data.

Enables real-time applications with strict causal constraints.

Abstract

With recent advances in embodied agents and AR devices, egocentric observations are readily available as input for real-world interactive online applications. However, egocentric viewpoints can only sporadically observe hands, in addition to the estimated head trajectory. We propose EgoForce, an online framework for reconstructing long-term full-body motion from noisy egocentric input. While existing generative frameworks can robustly handle noisy and sparse measurements, they assume a fixed-length observation window is available and are thus not suitable for real-time applications. Faster inference often relies on autoregressive prediction, sacrificing robustness. In contrast, we adopt a diffusion-based method with a temporally asymmetric noise schedule inspired by Diffusion Forcing. Specifically, our approach models temporally evolving uncertainty and incrementally denoises states as new streaming observations arrive. Combined with a noise-robust imputation strategy, EgoForce progressively generates stable and coherent full-body motion under strict causal constraints. Experiments demonstrate that our online framework outperforms existing online and offline methods, enabling long-horizon, full-body motion reconstruction in challenging egocentric scenarios.