Conservative Generator, Progressive Discriminator: Coordination of Adversaries in Few-shot Incremental Image Synthesis

TL;DR

This paper introduces ConPro, a novel GAN-based framework for incremental few-shot image synthesis, addressing catastrophic forgetting and data efficiency by coordinating a conservative generator and a progressive discriminator.

Contribution

The work proposes a new GAN framework with a conservative generator and a progressive discriminator for effective incremental few-shot image synthesis.

Findings

ConPro effectively preserves past knowledge during incremental learning.

The framework minimizes overfitting with limited data.

ConPro achieves improved forward transfer in incremental image synthesis.

Abstract

The capacity to learn incrementally from an online stream of data is an envied trait of human learners, as deep neural networks typically suffer from catastrophic forgetting and stability-plasticity dilemma. Several works have previously explored incremental few-shot learning, a task with greater challenges due to data constraint, mostly in classification setting with mild success. In this work, we study the underrepresented task of generative incremental few-shot learning. To effectively handle the inherent challenges of incremental learning and few-shot learning, we propose a novel framework named ConPro that leverages the two-player nature of GANs. Specifically, we design a conservative generator that preserves past knowledge in parameter and compute efficient manner, and a progressive discriminator that learns to reason semantic distances between past and present task samples, minimizing overfitting with few data points and pursuing good forward transfer. We present experiments to validate the effectiveness of ConPro.