Compositional Zero-Shot Learning with Contextualized Cues and Adaptive Contrastive Training

TL;DR

This paper proposes a novel framework for compositional zero-shot learning that enhances attribute-object understanding and linkage through sequential prediction and adaptive contrastive training, achieving state-of-the-art results.

Contribution

Introduces ULAO, a framework with modules for improved primitive understanding and attribute-object linkage using contrastive learning with hard negatives and adaptive loss.

Findings

State-of-the-art performance on three benchmarks

Effective attribute-object linkage in CZSL

Improved primitive understanding via sequential prediction

Abstract

Compositional Zero-Shot Learning (CZSL) aims to recognize unseen combinations of seen attributes and objects. Current CLIP-based methods in CZSL, despite their advancements, often fail to effectively understand and link the attributes and objects due to inherent limitations in CLIP's pretraining mechanisms. To address these shortcomings, this paper introduces a novel framework, Understanding and Linking Attributes and Objects (ULAO) in CZSL, which comprises two innovative modules. The Understanding Attributes and Objects (UAO) module improves primitive understanding by sequential primitive prediction and leveraging recognized objects as contextual hints for attribute classification. Concurrently, the Linking Attributes and Objects (LAO) module improves the attribute-object linkage understanding through a new contrastive learning strategy that incorporates tailored hard negative generation and adaptive loss adjustments. We demonstrate our model's superiority by showcasing its state-of-the-art performance across three benchmark datasets in both Closed-World (CW) and Open-World (OW) scenarios.