Discovering Spatial Relationships by Transformers for Domain Generalization

TL;DR

This paper introduces a hybrid transformer-based architecture to model spatial relationships in images, enhancing domain generalization by capturing both local features and their relationships, leading to state-of-the-art results.

Contribution

It proposes a novel hybrid model combining CNNs and self-attention to better capture spatial relationships for improved domain generalization.

Findings

Outperforms state-of-the-art by 2.2% on PACS

Outperforms state-of-the-art by 3.4% on Office-Home

Achieves superior domain generalization performance

Abstract

Due to the rapid increase in the diversity of image data, the problem of domain generalization has received increased attention recently. While domain generalization is a challenging problem, it has achieved great development thanks to the fast development of AI techniques in computer vision. Most of these advanced algorithms are proposed with deep architectures based on convolution neural nets (CNN). However, though CNNs have a strong ability to find the discriminative features, they do a poor job of modeling the relations between different locations in the image due to the response to CNN filters are mostly local. Since these local and global spatial relationships are characterized to distinguish an object under consideration, they play a critical role in improving the generalization ability against the domain gap. In order to get the object parts relationships to gain better domain generalization, this work proposes to use the self attention model. However, the attention models are proposed for sequence, which are not expert in discriminate feature extraction for 2D images. Considering this, we proposed a hybrid architecture to discover the spatial relationships between these local features, and derive a composite representation that encodes both the discriminative features and their relationships to improve the domain generalization. Evaluation on three well-known benchmarks demonstrates the benefits of modeling relationships between the features of an image using the proposed method and achieves state-of-the-art domain generalization performance. More specifically, the proposed algorithm outperforms the state-of-the-art by 2.2% and 3.4% on PACS and Office-Home databases, respectively.