# CLAREL: Classification via retrieval loss for zero-shot learning

## TL;DR

CLAREL introduces a novel instance-based deep metric learning method with semantic supervision for zero-shot learning, significantly improving fine-grained cross-modal classification performance, especially in generalized zero-shot settings.

## Contribution

It demonstrates that per-image semantic supervision enhances zero-shot performance and provides a probabilistic basis for metric rescaling in generalized zero-shot learning.

## Key findings

- Outperforms existing methods on CUB and FLOWERS datasets
- Improves zero-shot classification accuracy
- Addresses classifying unseen classes effectively

## Abstract

We address the problem of learning fine-grained cross-modal representations. We propose an instance-based deep metric learning approach in joint visual and textual space. The key novelty of this paper is that it shows that using per-image semantic supervision leads to substantial improvement in zero-shot performance over using class-only supervision. On top of that, we provide a probabilistic justification for a metric rescaling approach that solves a very common problem in the generalized zero-shot learning setting, i.e., classifying test images from unseen classes as one of the classes seen during training. We evaluate our approach on two fine-grained zero-shot learning datasets: CUB and FLOWERS. We find that on the generalized zero-shot classification task CLAREL consistently outperforms the existing approaches on both datasets.