Adversarial Training of Variational Auto-encoders for Continual Zero-shot Learning(A-CZSL)

TL;DR

This paper introduces A-CZSL, a hybrid variational auto-encoder model designed for continual zero-shot learning that mitigates catastrophic forgetting and effectively distinguishes unseen classes in sequential tasks.

Contribution

The paper proposes a novel hybrid VAE architecture with shared and task-specific modules for continual ZSL, addressing catastrophic forgetting without storing all previous data.

Findings

Outperforms baseline models on multiple datasets

Effective in class sequential learning for ZSL and GZSL

Maintains performance without large memory storage

Abstract

Most of the existing artificial neural networks(ANNs) fail to learn continually due to catastrophic forgetting, while humans can do the same by maintaining previous tasks' performances. Although storing all the previous data can alleviate the problem, it takes a large memory, infeasible in real-world utilization. We propose a continual zero-shot learning model(A-CZSL) that is more suitable in real-case scenarios to address the issue that can learn sequentially and distinguish classes the model has not seen during training. Further, to enhance the reliability, we develop A-CZSL for a single head continual learning setting where task identity is revealed during the training process but not during the testing. We present a hybrid network that consists of a shared VAE module to hold information of all tasks and task-specific private VAE modules for each task. The model's size grows with each task to prevent catastrophic forgetting of task-specific skills, and it includes a replay approach to preserve shared skills. We demonstrate our hybrid model outperforms the baselines and is effective on several datasets, i.e., CUB, AWA1, AWA2, and aPY. We show our method is superior in class sequentially learning with ZSL(Zero-Shot Learning) and GZSL(Generalized Zero-Shot Learning).