Adversarial Domain Adaptation for Action Recognition Around the Clock

TL;DR

This paper introduces 3D-DiNet, an adversarial domain adaptation model that improves nighttime action recognition by learning domain-invariant features, enabling effective unsupervised adaptation from daytime to nighttime videos.

Contribution

It proposes a novel adversarial domain adaptation framework for action recognition that combines supervised and unsupervised learning to handle cross-domain low-light scenarios.

Findings

Achieves state-of-the-art performance on InFAR dataset

Effectively learns domain-invariant features for nighttime action recognition

Enables end-to-end training with standard backpropagation

Abstract

Due to the numerous potential applications in visual surveillance and nighttime driving, recognizing human action in low-light conditions remains a difficult problem in computer vision. Existing methods separate action recognition and dark enhancement into two distinct steps to accomplish this task. However, isolating the recognition and enhancement impedes end-to-end learning of the space-time representation for video action classification. This paper presents a domain adaptation-based action recognition approach that uses adversarial learning in cross-domain settings to learn cross-domain action recognition. Supervised learning can train it on a large amount of labeled data from the source domain (daytime action sequences). However, it uses deep domain invariant features to perform unsupervised learning on many unlabelled data from the target domain (night-time action sequences). The resulting augmented model, named 3D-DiNet can be trained using standard backpropagation with an additional layer. It achieves SOTA performance on InFAR and XD145 actions datasets.