Continual Zero-Shot Learning through Semantically Guided Generative Random Walks

TL;DR

This paper introduces a novel semantically guided generative approach for continual zero-shot learning, leveraging theoretical insights and a new loss function to improve the generation of unseen class representations without access to unseen semantic data during training.

Contribution

It provides the first theoretical explanation for generative modeling benefits in CZSL and proposes a new GRW loss to enhance unseen class generation.

Findings

Achieves state-of-the-art results on multiple datasets

Surpasses existing CZSL methods by 3-7%

Introduces a theoretically motivated generative approach

Abstract

Learning novel concepts, remembering previous knowledge, and adapting it to future tasks occur simultaneously throughout a human's lifetime. To model such comprehensive abilities, continual zero-shot learning (CZSL) has recently been introduced. However, most existing methods overused unseen semantic information that may not be continually accessible in realistic settings. In this paper, we address the challenge of continual zero-shot learning where unseen information is not provided during training, by leveraging generative modeling. The heart of the generative-based methods is to learn quality representations from seen classes to improve the generative understanding of the unseen visual space. Motivated by this, we introduce generalization-bound tools and provide the first theoretical explanation for the benefits of generative modeling to CZSL tasks. Guided by the theoretical analysis, we then propose our learning algorithm that employs a novel semantically guided Generative Random Walk (GRW) loss. The GRW loss augments the training by continually encouraging the model to generate realistic and characterized samples to represent the unseen space. Our algorithm achieves state-of-the-art performance on AWA1, AWA2, CUB, and SUN datasets, surpassing existing CZSL methods by 3-7\%. The code has been made available here \url{https://github.com/wx-zhang/IGCZSL}