Siamese Contrastive Embedding Network for Compositional Zero-Shot Learning

TL;DR

This paper introduces a Siamese Contrastive Embedding Network that effectively recognizes unseen state-object compositions in visual data by embedding features into a space that separates state and object prototypes, enhancing robustness and accuracy.

Contribution

The paper proposes a novel Siamese Contrastive Embedding Network with a State Transition Module to improve compositional zero-shot learning by better modeling state-object interactions.

Findings

Significantly outperforms existing methods on benchmark datasets.

Effectively captures separate prototypes of states and objects.

Enhances robustness through increased diversity of training compositions.

Abstract

Compositional Zero-Shot Learning (CZSL) aims to recognize unseen compositions formed from seen state and object during training. Since the same state may be various in the visual appearance while entangled with different objects, CZSL is still a challenging task. Some methods recognize state and object with two trained classifiers, ignoring the impact of the interaction between object and state; the other methods try to learn the joint representation of the state-object compositions, leading to the domain gap between seen and unseen composition sets. In this paper, we propose a novel Siamese Contrastive Embedding Network (SCEN) (Code: https://github.com/XDUxyLi/SCEN-master) for unseen composition recognition. Considering the entanglement between state and object, we embed the visual feature into a Siamese Contrastive Space to capture prototypes of them separately, alleviating the interaction between state and object. In addition, we design a State Transition Module (STM) to increase the diversity of training compositions, improving the robustness of the recognition model. Extensive experiments indicate that our method significantly outperforms the state-of-the-art approaches on three challenging benchmark datasets, including the recent proposed C-QGA dataset.