Relation-aware Compositional Zero-shot Learning for Attribute-Object Pair Recognition

TL;DR

This paper introduces BMP-Net, a relation-aware model for recognizing unseen attribute-object pairs in images by learning rich primitive concept features and employing a blocking mechanism to generalize better.

Contribution

The paper presents BMP-Net, a novel relation-aware compositional zero-shot learning model with a blocking mechanism to improve unseen pair recognition.

Findings

Effective in recognizing unseen attribute-object pairs

Outperforms existing methods on benchmark datasets

Reduces bias towards seen concepts

Abstract

This paper proposes a novel model for recognizing images with composite attribute-object concepts, notably for composite concepts that are unseen during model training. We aim to explore the three key properties required by the task --- relation-aware, consistent, and decoupled --- to learn rich and robust features for primitive concepts that compose attribute-object pairs. To this end, we propose the Blocked Message Passing Network (BMP-Net). The model consists of two modules. The concept module generates semantically meaningful features for primitive concepts, whereas the visual module extracts visual features for attributes and objects from input images. A message passing mechanism is used in the concept module to capture the relations between primitive concepts. Furthermore, to prevent the model from being biased towards seen composite concepts and reduce the entanglement between attributes and objects, we propose a blocking mechanism that equalizes the information available to the model for both seen and unseen concepts. Extensive experiments and ablation studies on two benchmarks show the efficacy of the proposed model.