Dark Transformer: A Video Transformer for Action Recognition in the Dark

TL;DR

Dark Transformer is a novel video transformer model designed to improve action recognition in low-light conditions by leveraging spatiotemporal self-attention, achieving state-of-the-art results on multiple benchmarks.

Contribution

It introduces a cross-domain spatiotemporal self-attention mechanism in video transformers specifically for action recognition in dark environments, enabling end-to-end learning.

Findings

Achieves state-of-the-art performance on InFAR, XD145, and ARID datasets.

Effectively enhances action recognition accuracy in adverse lighting conditions.

Demonstrates practical potential for real-world applications in surveillance and nighttime driving.

Abstract

Recognizing human actions in adverse lighting conditions presents significant challenges in computer vision, with wide-ranging applications in visual surveillance and nighttime driving. Existing methods tackle action recognition and dark enhancement separately, limiting the potential for end-to-end learning of spatiotemporal representations for video action classification. This paper introduces Dark Transformer, a novel video transformer-based approach for action recognition in low-light environments. Dark Transformer leverages spatiotemporal self-attention mechanisms in cross-domain settings to enhance cross-domain action recognition. By extending video transformers to learn cross-domain knowledge, Dark Transformer achieves state-of-the-art performance on benchmark action recognition datasets, including InFAR, XD145, and ARID. The proposed approach demonstrates significant promise in addressing the challenges of action recognition in adverse lighting conditions, offering practical implications for real-world applications.