Open World Compositional Zero-Shot Learning

TL;DR

This paper introduces a method for compositional zero-shot learning in open world settings, estimating composition feasibility to improve recognition of unseen state-object pairs, and demonstrates significant performance gains over existing methods.

Contribution

The work proposes a novel feasibility scoring approach for open world CZSL, enhancing recognition accuracy without prior knowledge of unseen compositions.

Findings

Feasibility scores improve open world CZSL performance

State-of-the-art results in closed world scenarios

Performance degradation occurs without feasibility estimation

Abstract

Compositional Zero-Shot learning (CZSL) requires to recognize state-object compositions unseen during training. In this work, instead of assuming prior knowledge about the unseen compositions, we operate in the open world setting, where the search space includes a large number of unseen compositions some of which might be unfeasible. In this setting, we start from the cosine similarity between visual features and compositional embeddings. After estimating the feasibility score of each composition, we use these scores to either directly mask the output space or as a margin for the cosine similarity between visual features and compositional embeddings during training. Our experiments on two standard CZSL benchmarks show that all the methods suffer severe performance degradation when applied in the open world setting. While our simple CZSL model achieves state-of-the-art performances in the closed world scenario, our feasibility scores boost the performance of our approach in the open world setting, clearly outperforming the previous state of the art.