ProCC: Progressive Cross-primitive Compatibility for Open-World Compositional Zero-Shot Learning

TL;DR

ProCC introduces a progressive training method with a cross-primitive compatibility module that models interactions between state and object features, significantly improving open-world compositional zero-shot learning performance without external knowledge.

Contribution

The paper proposes a novel Progressive Cross-primitive Compatibility (ProCC) approach that explicitly models state-object interactions and employs a progressive training paradigm for OW-CZSL.

Findings

Outperforms existing methods on benchmark datasets

Effective in both OW-CZSL and pCZSL settings

Demonstrates large margin improvements

Abstract

Open-World Compositional Zero-shot Learning (OW-CZSL) aims to recognize novel compositions of state and object primitives in images with no priors on the compositional space, which induces a tremendously large output space containing all possible state-object compositions. Existing works either learn the joint compositional state-object embedding or predict simple primitives with separate classifiers. However, the former heavily relies on external word embedding methods, and the latter ignores the interactions of interdependent primitives, respectively. In this paper, we revisit the primitive prediction approach and propose a novel method, termed Progressive Cross-primitive Compatibility (ProCC), to mimic the human learning process for OW-CZSL tasks. Specifically, the cross-primitive compatibility module explicitly learns to model the interactions of state and object features with the trainable memory units, which efficiently acquires cross-primitive visual attention to reason high-feasibility compositions, without the aid of external knowledge. Moreover, considering the partial-supervision setting (pCZSL) as well as the imbalance issue of multiple task prediction, we design a progressive training paradigm to enable the primitive classifiers to interact to obtain discriminative information in an easy-to-hard manner. Extensive experiments on three widely used benchmark datasets demonstrate that our method outperforms other representative methods on both OW-CZSL and pCZSL settings by large margins.