Continuous-Time Human Motion Field from Events

TL;DR

This paper introduces a novel continuous-time human motion field estimation method from event streams, leveraging a neural network to predict motion as a time-implicit function, enabling high-resolution, efficient, and accurate human motion analysis.

Contribution

It presents the first continuous-time human motion field model from events using a time-implicit function, improving accuracy and efficiency over prior discrete-time approaches.

Findings

Improves joint errors by 23.8% on high-speed datasets

Reduces computational time by 69% compared to previous methods

Introduces a new high-speed human motion dataset with hardware synchronization

Abstract

This paper addresses the challenges of estimating a continuous-time human motion field from a stream of events. Existing Human Mesh Recovery (HMR) methods rely predominantly on frame-based approaches, which are prone to aliasing and inaccuracies due to limited temporal resolution and motion blur. In this work, we predict a continuous-time human motion field directly from events by leveraging a recurrent feed-forward neural network to predict human motion in the latent space of possible human motions. Prior state-of-the-art event-based methods rely on computationally intensive optimization across a fixed number of poses at high frame rates, which becomes prohibitively expensive as we increase the temporal resolution. In comparison, we present the first work that replaces traditional discrete-time predictions with a continuous human motion field represented as a time-implicit function, enabling parallel pose queries at arbitrary temporal resolutions. Despite the promises of event cameras, few benchmarks have tested the limit of high-speed human motion estimation. We introduce Beam-splitter Event Agile Human Motion Dataset-a hardware-synchronized high-speed human dataset to fill this gap. On this new data, our method improves joint errors by 23.8% compared to previous event human methods while reducing the computational time by 69%.