Dont Even Look Once: Synthesizing Features for Zero-Shot Detection

TL;DR

This paper introduces DELO, a novel zero-shot detection method that synthesizes features for unseen objects, significantly improving detection accuracy on large-scale datasets like Pascal VOC and MSCOCO.

Contribution

DELO is the first approach to synthesize visual features for unseen objects and integrate them into existing detection frameworks for improved zero-shot detection.

Findings

Significant accuracy improvements over vanilla detectors.

Effective synthesis of visual features for unseen objects.

Outperforms existing zero-shot detection methods.

Abstract

Zero-shot detection, namely, localizing both seen and unseen objects, increasingly gains importance for large-scale applications, with large number of object classes, since, collecting sufficient annotated data with ground truth bounding boxes is simply not scalable. While vanilla deep neural networks deliver high performance for objects available during training, unseen object detection degrades significantly. At a fundamental level, while vanilla detectors are capable of proposing bounding boxes, which include unseen objects, they are often incapable of assigning high-confidence to unseen objects, due to the inherent precision/recall tradeoffs that requires rejecting background objects. We propose a novel detection algorithm Dont Even Look Once (DELO), that synthesizes visual features for unseen objects and augments existing training algorithms to incorporate unseen object detection. Our proposed scheme is evaluated on Pascal VOC and MSCOCO, and we demonstrate significant improvements in test accuracy over vanilla and other state-of-art zero-shot detectors