Partial Video Domain Adaptation with Partial Adversarial Temporal Attentive Network

TL;DR

This paper introduces PATAN, a novel network that leverages spatial and temporal features with attention mechanisms to effectively filter source-only classes in partial video domain adaptation, achieving state-of-the-art results.

Contribution

The paper proposes a new Partial Adversarial Temporal Attentive Network (PATAN) that addresses PVDA by combining spatial and temporal features with attention for class filtering.

Findings

PATAN achieves state-of-the-art performance on multiple PVDA benchmarks.

The method effectively filters source-only classes using attention on local temporal features.

New PVDA benchmarks are introduced to facilitate future research.

Abstract

Partial Domain Adaptation (PDA) is a practical and general domain adaptation scenario, which relaxes the fully shared label space assumption such that the source label space subsumes the target one. The key challenge of PDA is the issue of negative transfer caused by source-only classes. For videos, such negative transfer could be triggered by both spatial and temporal features, which leads to a more challenging Partial Video Domain Adaptation (PVDA) problem. In this paper, we propose a novel Partial Adversarial Temporal Attentive Network (PATAN) to address the PVDA problem by utilizing both spatial and temporal features for filtering source-only classes. Besides, PATAN constructs effective overall temporal features by attending to local temporal features that contribute more toward the class filtration process. We further introduce new benchmarks to facilitate research on PVDA problems, covering a wide range of PVDA scenarios. Empirical results demonstrate the state-of-the-art performance of our proposed PATAN across the multiple PVDA benchmarks.