Exploring Few-Shot Adaptation for Activity Recognition on Diverse Domains

TL;DR

This paper introduces a new benchmark and a novel method called RelaMiX for few-shot domain adaptation in activity recognition, enabling effective model adaptation with minimal labeled data across diverse and challenging domains.

Contribution

It presents a comprehensive FSDA-AR benchmark with diverse datasets and proposes RelaMiX, a novel approach combining relational attention, feature mixing, and domain alignment for improved few-shot adaptation.

Findings

RelaMiX achieves state-of-the-art results across all datasets.

Few-shot adaptation performs comparably to unsupervised methods with fewer labels.

The benchmark encourages future research in diverse domain adaptation for activity recognition.

Abstract

Domain adaptation is essential for activity recognition to ensure accurate and robust performance across diverse environments, sensor types, and data sources. Unsupervised domain adaptation methods have been extensively studied, yet, they require large-scale unlabeled data from the target domain. In this work, we focus on Few-Shot Domain Adaptation for Activity Recognition (FSDA-AR), which leverages a very small amount of labeled target videos to achieve effective adaptation. This approach is appealing for applications because it only needs a few or even one labeled example per class in the target domain, ideal for recognizing rare but critical activities. However, the existing FSDA-AR works mostly focus on the domain adaptation on sports videos, where the domain diversity is limited. We propose a new FSDA-AR benchmark using five established datasets considering the adaptation on more diverse and challenging domains. Our results demonstrate that FSDA-AR performs comparably to unsupervised domain adaptation with significantly fewer labeled target domain samples. We further propose a novel approach, RelaMiX, to better leverage the few labeled target domain samples as knowledge guidance. RelaMiX encompasses a temporal relational attention network with relation dropout, alongside a cross-domain information alignment mechanism. Furthermore, it integrates a mechanism for mixing features within a latent space by using the few-shot target domain samples. The proposed RelaMiX solution achieves state-of-the-art performance on all datasets within the FSDA-AR benchmark. To encourage future research of few-shot domain adaptation for activity recognition, our code will be publicly available at https://github.com/KPeng9510/RelaMiX.