Event-based Egocentric Human Pose Estimation in Dynamic Environment

TL;DR

This paper introduces D-EventEgo, a novel framework for estimating human pose from front-facing event-based camera data, effectively handling dynamic environments and low-light conditions.

Contribution

The work presents the first framework for egocentric human pose estimation using event-based cameras, including a motion segmentation module to improve accuracy amidst dynamic objects.

Findings

Outperforms baseline in 4 out of 5 metrics in dynamic settings

Introduces a synthetic dataset derived from EgoBody for evaluation

Effectively handles low-light and motion blur scenarios

Abstract

Estimating human pose using a front-facing egocentric camera is essential for applications such as sports motion analysis, VR/AR, and AI for wearable devices. However, many existing methods rely on RGB cameras and do not account for low-light environments or motion blur. Event-based cameras have the potential to address these challenges. In this work, we introduce a novel task of human pose estimation using a front-facing event-based camera mounted on the head and propose D-EventEgo, the first framework for this task. The proposed method first estimates the head poses, and then these are used as conditions to generate body poses. However, when estimating head poses, the presence of dynamic objects mixed with background events may reduce head pose estimation accuracy. Therefore, we introduce the Motion Segmentation Module to remove dynamic objects and extract background information. Extensive experiments on our synthetic event-based dataset derived from EgoBody, demonstrate that our approach outperforms our baseline in four out of five evaluation metrics in dynamic environments.