Resolving Semantic Confusions for Improved Zero-Shot Detection

TL;DR

This paper introduces a novel generative approach with triplet and cyclic-consistency losses to reduce semantic confusion in zero-shot detection, significantly improving unseen object recognition.

Contribution

It proposes a new generative model training method that incorporates class dissimilarity and semantic consistency to enhance zero-shot detection performance.

Findings

Significant improvement on MSCOCO and PASCAL-VOC datasets

Reduced semantic confusion in zero-shot detection

Enhanced recognition of unseen classes

Abstract

Zero-shot detection (ZSD) is a challenging task where we aim to recognize and localize objects simultaneously, even when our model has not been trained with visual samples of a few target ("unseen") classes. Recently, methods employing generative models like GANs have shown some of the best results, where unseen-class samples are generated based on their semantics by a GAN trained on seen-class data, enabling vanilla object detectors to recognize unseen objects. However, the problem of semantic confusion still remains, where the model is sometimes unable to distinguish between semantically-similar classes. In this work, we propose to train a generative model incorporating a triplet loss that acknowledges the degree of dissimilarity between classes and reflects them in the generated samples. Moreover, a cyclic-consistency loss is also enforced to ensure that generated visual samples of a class highly correspond to their own semantics. Extensive experiments on two benchmark ZSD datasets - MSCOCO and PASCAL-VOC - demonstrate significant gains over the current ZSD methods, reducing semantic confusion and improving detection for the unseen classes.