KG-SP: Knowledge Guided Simple Primitives for Open World Compositional Zero-Shot Learning

TL;DR

This paper introduces KG-SP, a simple yet effective approach for open-world compositional zero-shot learning that predicts states and objects independently, uses external knowledge to filter unfeasible compositions, and extends to partial supervision scenarios.

Contribution

The paper proposes a novel knowledge-guided primitive prediction method for OW-CZSL and pCZSL, improving state-of-the-art performance with a simple, interpretable model.

Findings

Achieves state-of-the-art results in OW-CZSL and pCZSL

Effectively filters unfeasible compositions using external knowledge

Performs well even with partial supervision and semi-supervised techniques

Abstract

The goal of open-world compositional zero-shot learning (OW-CZSL) is to recognize compositions of state and objects in images, given only a subset of them during training and no prior on the unseen compositions. In this setting, models operate on a huge output space, containing all possible state-object compositions. While previous works tackle the problem by learning embeddings for the compositions jointly, here we revisit a simple CZSL baseline and predict the primitives, i.e. states and objects, independently. To ensure that the model develops primitive-specific features, we equip the state and object classifiers with separate, non-linear feature extractors. Moreover, we estimate the feasibility of each composition through external knowledge, using this prior to remove unfeasible compositions from the output space. Finally, we propose a new setting, i.e. CZSL under partial supervision (pCZSL), where either only objects or state labels are available during training, and we can use our prior to estimate the missing labels. Our model, Knowledge-Guided Simple Primitives (KG-SP), achieves state of the art in both OW-CZSL and pCZSL, surpassing most recent competitors even when coupled with semi-supervised learning techniques. Code available at: https://github.com/ExplainableML/KG-SP.