CAMS: Towards Compositional Zero-Shot Learning via Gated Cross-Attention and Multi-Space Disentanglement

TL;DR

CAMS introduces a novel approach for compositional zero-shot learning by extracting fine-grained semantic features through gated cross-attention and multi-space disentanglement, significantly improving generalization to unseen attribute-object pairs.

Contribution

The paper proposes CAMS, a method that enhances CZSL by extracting detailed semantic features and disentangling attribute and object representations in multiple spaces, surpassing previous methods.

Findings

Achieves state-of-the-art results on MIT-States, UT-Zappos, and C-GQA benchmarks.

Effectively disentangles attribute and object semantics for better unseen composition recognition.

Demonstrates improved generalization in both closed-world and open-world CZSL settings.

Abstract

Compositional zero-shot learning (CZSL) aims to learn the concepts of attributes and objects in seen compositions and to recognize their unseen compositions. Most Contrastive Language-Image Pre-training (CLIP)-based CZSL methods focus on disentangling attributes and objects by leveraging the global semantic representation obtained from the image encoder. However, this representation has limited representational capacity and do not allow for complete disentanglement of the two. To this end, we propose CAMS, which aims to extract semantic features from visual features and perform semantic disentanglement in multidimensional spaces, thereby improving generalization over unseen attribute-object compositions. Specifically, CAMS designs a Gated Cross-Attention that captures fine-grained semantic features from the high-level image encoding blocks of CLIP through a set of latent units, while adaptively suppressing background and other irrelevant information. Subsequently, it conducts Multi-Space Disentanglement to achieve disentanglement of attribute and object semantics. Experiments on three popular benchmarks (MIT-States, UT-Zappos, and C-GQA) demonstrate that CAMS achieves state-of-the-art performance in both closed-world and open-world settings. The code is available at https://github.com/ybyangjing/CAMS.