Semantically Guided Action Anticipation

TL;DR

This paper introduces a novel semantic-guided approach for unsupervised domain adaptation that aligns relative class relationships rather than absolute features, improving transfer performance across diverse datasets.

Contribution

It proposes a new method focusing on aligning semantic relationships in latent space, outperforming previous domain adaptation techniques across multiple datasets.

Findings

Achieves higher accuracy in four diverse datasets.

Surpasses previous methods in 18 adaptation scenarios.

Improves average accuracy by up to 5.75%.

Abstract

Unsupervised domain adaptation remains a critical challenge in enabling the knowledge transfer of models across unseen domains. Existing methods struggle to balance the need for domain-invariant representations with preserving domain-specific features, which is often due to alignment approaches that impose the projection of samples with similar semantics close in the latent space despite their drastic domain differences. We introduce a novel approach that shifts the focus from aligning representations in absolute coordinates to aligning the relative positioning of equivalent concepts in latent spaces. Our method defines a domain-agnostic structure upon the semantic/geometric relationships between class labels in language space and guides adaptation, ensuring that the organization of samples in visual space reflects reference inter-class relationships while preserving domain-specific characteristics. We empirically demonstrate our method's superiority in domain adaptation tasks across four diverse image and video datasets. Remarkably, we surpass previous works in 18 different adaptation scenarios across four diverse image and video datasets with average accuracy improvements of +3.32% on DomainNet, +5.75% in GeoPlaces, +4.77% on GeoImnet, and +1.94% mean class accuracy improvement on EgoExo4D.