Improving Semantic Embedding Consistency by Metric Learning for Zero-Shot Classification

TL;DR

This paper introduces a metric learning approach to improve semantic embedding consistency in zero-shot image classification, enabling better recognition without class labels during training.

Contribution

It formulates zero-shot learning as a metric learning problem using image-attribute pairs, removing the need for class labels during training.

Findings

Achieves state-of-the-art results on four zero-shot recognition datasets.

Effectively predicts image-attribute consistency at test time.

Simplifies zero-shot learning by avoiding class-based training constraints.

Abstract

This paper addresses the task of zero-shot image classification. The key contribution of the proposed approach is to control the semantic embedding of images -- one of the main ingredients of zero-shot learning -- by formulating it as a metric learning problem. The optimized empirical criterion associates two types of sub-task constraints: metric discriminating capacity and accurate attribute prediction. This results in a novel expression of zero-shot learning not requiring the notion of class in the training phase: only pairs of image/attributes, augmented with a consistency indicator, are given as ground truth. At test time, the learned model can predict the consistency of a test image with a given set of attributes , allowing flexible ways to produce recognition inferences. Despite its simplicity, the proposed approach gives state-of-the-art results on four challenging datasets used for zero-shot recognition evaluation.