PoliTO-IIT-CINI Submission to the EPIC-KITCHENS-100 Unsupervised Domain Adaptation Challenge for Action Recognition

TL;DR

This paper presents a comprehensive approach for unsupervised domain adaptation in action recognition within the EPIC-Kitchens-100 challenge, combining domain generalization, adversarial alignment, and ensemble methods to handle multiple shifts.

Contribution

It introduces a novel framework that extends domain generalization techniques to unsupervised settings and addresses environmental and temporal shifts in video action recognition.

Findings

Achieved 2nd place in verb recognition and 3rd in noun and action categories.

Effectively handled environmental and temporal domain shifts.

Enhanced video action recognition through multi-source and ensemble adaptation.

Abstract

In this report, we describe the technical details of our submission to the EPIC-Kitchens-100 Unsupervised Domain Adaptation (UDA) Challenge in Action Recognition. To tackle the domain-shift which exists under the UDA setting, we first exploited a recent Domain Generalization (DG) technique, called Relative Norm Alignment (RNA). Secondly, we extended this approach to work on unlabelled target data, enabling a simpler adaptation of the model to the target distribution in an unsupervised fashion. To this purpose, we included in our framework UDA algorithms, such as multi-level adversarial alignment and attentive entropy. By analyzing the challenge setting, we notice the presence of a secondary concurrence shift in the data, which is usually called environmental bias. It is caused by the existence of different environments, i.e., kitchens. To deal with these two shifts (environmental and temporal), we extended our system to perform Multi-Source Multi-Target Domain Adaptation. Finally, we employed distinct models in our final proposal to leverage the potential of popular video architectures, and we introduced two more losses for the ensemble adaptation. Our submission (entry 'plnet') is visible on the leaderboard and ranked in 2nd position for 'verb', and in 3rd position for both 'noun' and 'action'.