Model-agnostic Multi-Domain Learning with Domain-Specific Adapters for Action Recognition

TL;DR

This paper introduces a model-agnostic multi-domain learning approach for action recognition that uses domain-specific adapters to improve universality and efficiency across multiple datasets.

Contribution

It proposes a novel adapter-based method that switches both classification heads and feature adapters, enhancing multi-domain learning without assuming specific model structures.

Findings

Outperforms multi-head architectures in accuracy.

More efficient than training separate models for each domain.

Effective across multiple popular action recognition datasets.

Abstract

In this paper, we propose a multi-domain learning model for action recognition. The proposed method inserts domain-specific adapters between layers of domain-independent layers of a backbone network. Unlike a multi-head network that switches classification heads only, our model switches not only the heads, but also the adapters for facilitating to learn feature representations universal to multiple domains. Unlike prior works, the proposed method is model-agnostic and doesn't assume model structures unlike prior works. Experimental results on three popular action recognition datasets (HMDB51, UCF101, and Kinetics-400) demonstrate that the proposed method is more effective than a multi-head architecture and more efficient than separately training models for each domain.