Unified Framework for Open-World Compositional Zero-shot Learning

TL;DR

This paper presents a unified framework for open-world compositional zero-shot learning that enhances image-text interactions, reduces inference complexity, and combines joint and independent learning to improve recognition of novel compositions.

Contribution

It introduces a novel hybrid learning approach and a module to reduce inference complexity, achieving state-of-the-art results in OW-CZSL and surpassing large vision-language models.

Findings

Achieves state-of-the-art performance on three datasets.

Surpasses large vision-language models on two datasets.

Enhances inter-modality interactions for better recognition.

Abstract

Open-World Compositional Zero-Shot Learning (OW-CZSL) addresses the challenge of recognizing novel compositions of known primitives and entities. Even though prior works utilize language knowledge for recognition, such approaches exhibit limited interactions between language-image modalities. Our approach primarily focuses on enhancing the inter-modality interactions through fostering richer interactions between image and textual data. Additionally, we introduce a novel module aimed at alleviating the computational burden associated with exhaustive exploration of all possible compositions during the inference stage. While previous methods exclusively learn compositions jointly or independently, we introduce an advanced hybrid procedure that leverages both learning mechanisms to generate final predictions. Our proposed model, achieves state-of-the-art in OW-CZSL in three datasets, while surpassing Large Vision Language Models (LLVM) in two datasets.