DarkShake-DVS: Event-based Human Action Recognition under Low-light andShaking Camera Conditions

TL;DR

This paper introduces DarkShake-DVS, a large-scale benchmark for event-based human action recognition in low-light and shaky camera conditions, along with a novel method combining motion compensation and hybrid feature extraction.

Contribution

It presents the first comprehensive benchmark dataset and a new hybrid architecture for robust action recognition under challenging low-light and motion conditions.

Findings

EIS-HAR outperforms existing methods on multiple datasets.

DarkShake-DVS contains 18,041 real-world clips with synchronized IMU data.

The proposed approach effectively compensates for motion blur and improves recognition accuracy.

Abstract

Human Action Recognition (HAR) is a fundamental computer vision task with diverse real-world applications. Practical deployments often involve low-light environments and unconstrained 6-DoF camera motion, conditions that degrade visual quality, disrupt temporal coherence, and compromise reliability of existing methods. Event cameras, with high low-light sensitivity and microsecond-level temporal resolution, paired with an inertial measurement unit (IMU), present a promising solution. However, current research faces two key challenges: absence of a benchmark integrating low-light conditions, 6-DoF motion, and synchronized IMU data; and lack of effective motion compensation techniques. To address these, we propose Event-IMU Stabilized HAR (EIS-HAR), with two modules. The first is an EIS module that reduces motion blur via a non-linear warping function to reconstruct a motion-compensated input. The second is a HAR module with a four-stage hybrid architecture to efficiently extract spatiotemporal features for accurate action recognition. To alleviate data scarcity, we introduce DarkShake-DVS, the first large-scale event-based HAR benchmark that includes 18,041 realworld clips captured in low light and intense 6-DoF motion, supplemented by synchronized IMU data. Extensive experiments on three datasets demonstrate consistent superiority of EIS-HAR over state-of-the-art methods.