Zero-Shot Recognition through Image-Guided Semantic Classification

TL;DR

This paper introduces Image-Guided Semantic Classification (IGSC), a novel zero-shot learning framework that generates image-adaptive classifiers during inference, significantly improving performance over existing embedding-based methods.

Contribution

The paper proposes a new inverse approach to zero-shot learning by generating semantic classifiers from images, enhancing adaptability and accuracy over prior methods.

Findings

IGSC outperforms state-of-the-art embedding-based ZSL methods on benchmarks.

Semantic classifiers are image-adaptive and generated during inference.

The approach is simple to implement with minor modifications to existing architectures.

Abstract

We present a new embedding-based framework for zero-shot learning (ZSL). Most embedding-based methods aim to learn the correspondence between an image classifier (visual representation) and its class prototype (semantic representation) for each class. Motivated by the binary relevance method for multi-label classification, we propose to inversely learn the mapping between an image and a semantic classifier. Given an input image, the proposed Image-Guided Semantic Classification (IGSC) method creates a label classifier, being applied to all label embeddings to determine whether a label belongs to the input image. Therefore, semantic classifiers are image-adaptive and are generated during inference. IGSC is conceptually simple and can be realized by a slight enhancement of an existing deep architecture for classification; yet it is effective and outperforms state-of-the-art embedding-based generalized ZSL approaches on standard benchmarks.