CIZSL++: Creativity Inspired Generative Zero-Shot Learning

TL;DR

This paper introduces CIZSL++, a novel generative zero-shot learning framework inspired by human creativity, which improves recognition of unseen categories by exploring their visual features through creative hallucination and semantic guidance.

Contribution

The paper proposes two versions of CIZSL, integrating creativity-inspired hallucination and semantic-guided discrimination to enhance zero-shot learning performance.

Findings

CIZSL improves generalized ZSL on CUB and NABirds datasets.

CIZSL-v2 outperforms CIZSL-v1 in zero-shot tasks.

The approach benefits attribute-based ZSL on multiple datasets.

Abstract

Zero-shot learning (ZSL) aims at understanding unseen categories with no training examples from class-level descriptions. To improve the discriminative power of ZSL, we model the visual learning process of unseen categories with inspiration from the psychology of human creativity for producing novel art. First, we propose CIZSL-v1 as a creativity inspired model for generative ZSL. We relate ZSL to human creativity by observing that ZSL is about recognizing the unseen, and creativity is about creating a likable unseen. We introduce a learning signal inspired by creativity literature that explores the unseen space with hallucinated class-descriptions and encourages careful deviation of their visual feature generations from seen classes while allowing knowledge transfer from seen to unseen classes. Second, CIZSL-v2 is proposed as an improved version of CIZSL-v1 for generative zero-shot learning. CIZSL-v2 consists of an investigation of additional inductive losses for unseen classes along with a semantic guided discriminator. Empirically, we show consistently that CIZSL losses can improve generative ZSL models on the challenging task of generalized ZSL from a noisy text on CUB and NABirds datasets. We also show the advantage of our approach to Attribute-based ZSL on AwA2, aPY, and SUN datasets. We also show that CIZSL-v2 has improved performance compared to CIZSL-v1.