SEPose: A Synthetic Event-based Human Pose Estimation Dataset for Pedestrian Monitoring

TL;DR

SEPose is a large synthetic dataset of event-based human poses generated in simulation, designed to improve pedestrian monitoring systems under challenging conditions by enabling better model training and generalization.

Contribution

We introduce SEPose, a comprehensive synthetic event-based human pose dataset with nearly 350K annotated pedestrians, filling a data gap for training and evaluating pedestrian perception models.

Findings

Models trained on SEPose generalize well to real event-based data.

SEPose covers diverse environments, lighting, and weather conditions.

State-of-the-art models achieve promising results when trained on our dataset.

Abstract

Event-based sensors have emerged as a promising solution for addressing challenging conditions in pedestrian and traffic monitoring systems. Their low-latency and high dynamic range allow for improved response time in safety-critical situations caused by distracted walking or other unusual movements. However, the availability of data covering such scenarios remains limited. To address this gap, we present SEPose -- a comprehensive synthetic event-based human pose estimation dataset for fixed pedestrian perception generated using dynamic vision sensors in the CARLA simulator. With nearly 350K annotated pedestrians with body pose keypoints from the perspective of fixed traffic cameras, SEPose is a comprehensive synthetic multi-person pose estimation dataset that spans busy and light crowds and traffic across diverse lighting and weather conditions in 4-way intersections in urban, suburban, and rural environments. We train existing state-of-the-art models such as RVT and YOLOv8 on our dataset and evaluate them on real event-based data to demonstrate the sim-to-real generalization capabilities of the proposed dataset.