Structure propagation for zero-shot learning

TL;DR

This paper introduces structure propagation, a method that leverages both image and semantic class structures to improve zero-shot learning performance, effectively addressing the challenge of unseen class relationships.

Contribution

It proposes a novel structure propagation approach that jointly models image and semantic class manifolds for enhanced zero-shot classification.

Findings

Achieves promising results on AwA, CUB, Dogs, and SUN datasets.

Effectively models relationships between unseen classes.

Balances influence of manifold structures in compatibility learning.

Abstract

The key of zero-shot learning (ZSL) is how to find the information transfer model for bridging the gap between images and semantic information (texts or attributes). Existing ZSL methods usually construct the compatibility function between images and class labels with the consideration of the relevance on the semantic classes (the manifold structure of semantic classes). However, the relationship of image classes (the manifold structure of image classes) is also very important for the compatibility model construction. It is difficult to capture the relationship among image classes due to unseen classes, so that the manifold structure of image classes often is ignored in ZSL. To complement each other between the manifold structure of image classes and that of semantic classes information, we propose structure propagation (SP) for improving the performance of ZSL for classification. SP can jointly consider the manifold structure of image classes and that of semantic classes for approximating to the intrinsic structure of object classes. Moreover, the SP can describe the constrain condition between the compatibility function and these manifold structures for balancing the influence of the structure propagation iteration. The SP solution provides not only unseen class labels but also the relationship of two manifold structures that encode the positive transfer in structure propagation. Experimental results demonstrate that SP can attain the promising results on the AwA, CUB, Dogs and SUN databases.