Ontology-aware Network for Zero-shot Sketch-based Image Retrieval

TL;DR

This paper introduces an Ontology-Aware Network for zero-shot sketch-based image retrieval that effectively maintains inter-class distinctions and modality-specific features, outperforming existing methods on challenging datasets.

Contribution

The proposed OAN incorporates a novel inter-class independence learning mechanism and a distillation-based consistency preservation to enhance ZSSBIR performance.

Findings

Superior accuracy on Sketchy dataset

Effective preservation of modality-specific information

Enhanced inter-class distinction

Abstract

Zero-Shot Sketch-Based Image Retrieval (ZSSBIR) is an emerging task. The pioneering work focused on the modal gap but ignored inter-class information. Although recent work has begun to consider the triplet-based or contrast-based loss to mine inter-class information, positive and negative samples need to be carefully selected, or the model is prone to lose modality-specific information. To respond to these issues, an Ontology-Aware Network (OAN) is proposed. Specifically, the smooth inter-class independence learning mechanism is put forward to maintain inter-class peculiarity. Meanwhile, distillation-based consistency preservation is utilized to keep modality-specific information. Extensive experiments have demonstrated the superior performance of our algorithm on two challenging Sketchy and Tu-Berlin datasets.