Learning Invariant Visual Representations for Compositional Zero-Shot Learning

TL;DR

This paper introduces an invariant feature learning framework for compositional zero-shot learning, focusing on domain-invariant features to improve recognition of novel attribute-object pairs, outperforming previous methods.

Contribution

The paper proposes a novel invariant feature learning approach that aligns representations and gradients across domains to enhance compositional zero-shot learning.

Findings

Significant performance improvement over state-of-the-art methods

Effective learning of attribute- and object-invariant features

Robust generalization to unseen attribute-object compositions

Abstract

Compositional Zero-Shot Learning (CZSL) aims to recognize novel compositions using knowledge learned from seen attribute-object compositions in the training set. Previous works mainly project an image and a composition into a common embedding space to measure their compatibility score. However, both attributes and objects share the visual representations learned above, leading the model to exploit spurious correlations and bias towards seen pairs. Instead, we reconsider CZSL as an out-of-distribution generalization problem. If an object is treated as a domain, we can learn object-invariant features to recognize the attributes attached to any object reliably. Similarly, attribute-invariant features can also be learned when recognizing the objects with attributes as domains. Specifically, we propose an invariant feature learning framework to align different domains at the representation and gradient levels to capture the intrinsic characteristics associated with the tasks. Experiments on two CZSL benchmarks demonstrate that the proposed method significantly outperforms the previous state-of-the-art.