Zero-Shot Compositional Concept Learning

TL;DR

This paper introduces EpiCA, a novel zero-shot learning model that uses cross-attention and episode-based training to recognize unseen compositional concepts more effectively.

Contribution

The paper proposes an episode-based cross-attention network with a two-phase training strategy for improved zero-shot compositional concept recognition.

Findings

EpiCA outperforms recent approaches on ZSCL benchmarks.

The model effectively utilizes unlabeled test data in transductive training.

EpiCA improves recognition of novel attribute-object combinations.

Abstract

In this paper, we study the problem of recognizing compositional attribute-object concepts within the zero-shot learning (ZSL) framework. We propose an episode-based cross-attention (EpiCA) network which combines merits of cross-attention mechanism and episode-based training strategy to recognize novel compositional concepts. Firstly, EpiCA bases on cross-attention to correlate concept-visual information and utilizes the gated pooling layer to build contextualized representations for both images and concepts. The updated representations are used for a more in-depth multi-modal relevance calculation for concept recognition. Secondly, a two-phase episode training strategy, especially the transductive phase, is adopted to utilize unlabeled test examples to alleviate the low-resource learning problem. Experiments on two widely-used zero-shot compositional learning (ZSCL) benchmarks have demonstrated the effectiveness of the model compared with recent approaches on both conventional and generalized ZSCL settings.