Conditional Cross Attention Network for Multi-Space Embedding without Entanglement in Only a SINGLE Network

TL;DR

This paper introduces a Conditional Cross-Attention Network that creates disentangled multi-space embeddings for multiple object attributes within a single model, enhancing fine-grained image retrieval.

Contribution

It proposes a novel cross-attention mechanism combined with vision transformers to achieve attribute disentanglement in a unified network, improving robustness across datasets.

Findings

Achieved state-of-the-art results on multiple benchmark datasets.

Demonstrated effective attribute disentanglement through visualization.

First application of vision transformers to fine-grained image retrieval.

Abstract

Many studies in vision tasks have aimed to create effective embedding spaces for single-label object prediction within an image. However, in reality, most objects possess multiple specific attributes, such as shape, color, and length, with each attribute composed of various classes. To apply models in real-world scenarios, it is essential to be able to distinguish between the granular components of an object. Conventional approaches to embedding multiple specific attributes into a single network often result in entanglement, where fine-grained features of each attribute cannot be identified separately. To address this problem, we propose a Conditional Cross-Attention Network that induces disentangled multi-space embeddings for various specific attributes with only a single backbone. Firstly, we employ a cross-attention mechanism to fuse and switch the information of conditions (specific attributes), and we demonstrate its effectiveness through a diverse visualization example. Secondly, we leverage the vision transformer for the first time to a fine-grained image retrieval task and present a simple yet effective framework compared to existing methods. Unlike previous studies where performance varied depending on the benchmark dataset, our proposed method achieved consistent state-of-the-art performance on the FashionAI, DARN, DeepFashion, and Zappos50K benchmark datasets.