Separated Inter/Intra-Modal Fusion Prompts for Compositional Zero-Shot Learning

TL;DR

This paper introduces a novel prompt learning approach with inter/intra-modality fusion to enhance compositional zero-shot learning, effectively recognizing subtle semantic differences and object combinations.

Contribution

It proposes a new method combining inter/intra-modality fusion prompts to improve attribute recognition in CZSL, addressing limitations of previous prompt-based techniques.

Findings

Improved attribute recognition accuracy in CZSL tasks.

Effective handling of subtle semantic differences.

Enhanced recognition of multiple object compositions.

Abstract

Compositional Zero-Shot Learning (CZSL) aims to recognize subtle differences in meaning or the combination of states and objects through the use of known and unknown concepts during training. Existing methods either focused on prompt configuration or on using prompts to tune the pre-trained Vision-Language model. However, these methods faced challenges in accurately identifying subtle differences in meaning or combining states with objects. To jointly eradicate the above issues and construct an efficient and effective CZSL technique, we suggest a method to improve attribute recognition performance by utilizing diverse Prompt Learning with an Inter/Intra-Modality Fusion Synthesizer in scene understanding involving subtle semantic differences and multiple objects.