EIANet: A Novel Domain Adaptation Approach to Maximize Class Distinction with Neural Collapse Principles

TL;DR

EIANet introduces an attention-based neural collapse approach with ETF classifiers to improve class separation in source-free domain adaptation, especially for fine-grained tasks, leading to state-of-the-art results.

Contribution

The paper proposes a novel ETF-Informed Attention Network that enhances class prototype separation using neural collapse principles for improved source-free domain adaptation.

Findings

Achieves state-of-the-art performance on four SFDA datasets.

Effectively enlarges feature differences between classes.

Prevents misclassification of similar categories.

Abstract

Source-free domain adaptation (SFDA) aims to transfer knowledge from a labelled source domain to an unlabelled target domain. A major challenge in SFDA is deriving accurate categorical information for the target domain, especially when sample embeddings from different classes appear similar. This issue is particularly pronounced in fine-grained visual categorization tasks, where inter-class differences are subtle. To overcome this challenge, we introduce a novel ETF-Informed Attention Network (EIANet) to separate class prototypes by utilizing attention and neural collapse principles. More specifically, EIANet employs a simplex Equiangular Tight Frame (ETF) classifier in conjunction with an attention mechanism, facilitating the model to focus on discriminative features and ensuring maximum class prototype separation. This innovative approach effectively enlarges the feature difference between different classes in the latent space by locating salient regions, thereby preventing the misclassification of similar but distinct category samples and providing more accurate categorical information to guide the fine-tuning process on the target domain. Experimental results across four SFDA datasets validate EIANet's state-of-the-art performance. Code is available at: https://github.com/zichengpan/EIANet.