Discriminative Image Generation with Diffusion Models for Zero-Shot Learning

TL;DR

This paper introduces DIG-ZSL, a novel framework that generates discriminative images for unseen classes in zero-shot learning using text prompts and a class discrimination model, improving performance and interpretability.

Contribution

The paper proposes a new discriminative image generation method for zero-shot learning that does not rely on human-annotated prototypes, enhancing scalability and interpretability.

Findings

Generates diverse, high-quality images for unseen classes.

Outperforms previous non-annotated prototype methods significantly.

Achieves comparable or better results than methods using human-annotated prototypes.

Abstract

Generative Zero-Shot Learning (ZSL) methods synthesize class-related features based on predefined class semantic prototypes, showcasing superior performance. However, this feature generation paradigm falls short of providing interpretable insights. In addition, existing approaches rely on semantic prototypes annotated by human experts, which exhibit a significant limitation in their scalability to generalized scenes. To overcome these deficiencies, a natural solution is to generate images for unseen classes using text prompts. To this end, We present DIG-ZSL, a novel Discriminative Image Generation framework for Zero-Shot Learning. Specifically, to ensure the generation of discriminative images for training an effective ZSL classifier, we learn a discriminative class token (DCT) for each unseen class under the guidance of a pre-trained category discrimination model (CDM). Harnessing DCTs, we can generate diverse and high-quality images, which serve as informative unseen samples for ZSL tasks. In this paper, the extensive experiments and visualizations on four datasets show that our DIG-ZSL: (1) generates diverse and high-quality images, (2) outperforms previous state-of-the-art nonhuman-annotated semantic prototype-based methods by a large margin, and (3) achieves comparable or better performance than baselines that leverage human-annotated semantic prototypes. The codes will be made available upon acceptance of the paper.