Memory Efficient Temporal & Visual Graph Model for Unsupervised Video Domain Adaptation

TL;DR

This paper introduces a memory-efficient graph-based approach for unsupervised video domain adaptation that models each video as a single graph, reducing memory costs while maintaining superior performance.

Contribution

It proposes a novel graph attention network that models each video as a single graph, avoiding the need to store multiple frame combinations, thus significantly reducing memory usage.

Findings

Achieves superior performance compared to state-of-the-art methods.

Reduces memory cost substantially while maintaining accuracy.

Effectively aligns source and target videos in a domain-invariant feature space.

Abstract

Existing video domain adaption (DA) methods need to store all temporal combinations of video frames or pair the source and target videos, which are memory cost expensive and can't scale up to long videos. To address these limitations, we propose a memory-efficient graph-based video DA approach as follows. At first our method models each source or target video by a graph: nodes represent video frames and edges represent the temporal or visual similarity relationship between frames. We use a graph attention network to learn the weight of individual frames and simultaneously align the source and target video into a domain-invariant graph feature space. Instead of storing a large number of sub-videos, our method only constructs one graph with a graph attention mechanism for one video, reducing the memory cost substantially. The extensive experiments show that, compared with the state-of-art methods, we achieved superior performance while reducing the memory cost significantly.