A Multi-class Approach -- Building a Visual Classifier based on Textual Descriptions using Zero-Shot Learning

TL;DR

This paper introduces a multi-class visual classifier that leverages zero-shot learning and textual descriptions to recognize unseen classes, addressing data scarcity and domain constraints in image classification.

Contribution

It presents a novel multi-class zero-shot learning approach that uses textual descriptions for training, enabling recognition of unseen classes without additional labeled images.

Findings

Effective recognition of unseen classes using textual descriptions

Improved classification accuracy over binary ZSL models

Demonstrated applicability to diverse image datasets

Abstract

Machine Learning (ML) techniques for image classification routinely require many labelled images for training the model and while testing, we ought to use images belonging to the same domain as those used for training. In this paper, we overcome the two main hurdles of ML, i.e. scarcity of data and constrained prediction of the classification model. We do this by introducing a visual classifier which uses a concept of transfer learning, namely Zero-Shot Learning (ZSL), and standard Natural Language Processing techniques. We train a classifier by mapping labelled images to their textual description instead of training it for specific classes. Transfer learning involves transferring knowledge across domains that are similar. ZSL intelligently applies the knowledge learned while training for future recognition tasks. ZSL differentiates classes as two types: seen and unseen classes. Seen classes are the classes upon which we have trained our model and unseen classes are the classes upon which we test our model. The examples from unseen classes have not been encountered in the training phase. Earlier research in this domain focused on developing a binary classifier but, in this paper, we present a multi-class classifier with a Zero-Shot Learning approach.