# Generative Model for Zero-Shot Sketch-Based Image Retrieval

## TL;DR

This paper introduces a probabilistic generative model for zero-shot sketch-based image retrieval that generates images conditioned on sketches of unseen classes, transforming the retrieval task into an image-to-image search problem.

## Contribution

The paper proposes a novel generative model using inverse auto-regressive flow variational autoencoders for zero-shot SBIR, enabling effective retrieval on unseen classes.

## Key findings

- Significantly outperforms baselines on Sketchy dataset.
- Achieves robust image generation conditioned on novel sketches.
- Effective on TU Berlin dataset with novel class splits.

## Abstract

We present a probabilistic model for Sketch-Based Image Retrieval (SBIR) where, at retrieval time, we are given sketches from novel classes, that were not present at training time. Existing SBIR methods, most of which rely on learning class-wise correspondences between sketches and images, typically work well only for previously seen sketch classes, and result in poor retrieval performance on novel classes. To address this, we propose a generative model that learns to generate images, conditioned on a given novel class sketch. This enables us to reduce the SBIR problem to a standard image-to-image search problem. Our model is based on an inverse auto-regressive flow based variational autoencoder, with a feedback mechanism to ensure robust image generation. We evaluate our model on two very challenging datasets, Sketchy, and TU Berlin, with novel train-test split. The proposed approach significantly outperforms various baselines on both the datasets.