LFS-GAN: Lifelong Few-Shot Image Generation

TL;DR

LFS-GAN introduces a lifelong few-shot image generation framework that effectively prevents catastrophic forgetting and mode collapse, producing high-quality, diverse images across multiple tasks with limited data.

Contribution

The paper proposes LeFT, a novel task-specific modulator, and a mode seeking loss, enabling high-fidelity, diverse image generation in lifelong few-shot learning without forgetting.

Findings

Achieves state-of-the-art results in lifelong few-shot image generation.

Outperforms existing few-shot GANs even in single-task scenarios.

Generates high-quality, diverse images across various domains.

Abstract

We address a challenging lifelong few-shot image generation task for the first time. In this situation, a generative model learns a sequence of tasks using only a few samples per task. Consequently, the learned model encounters both catastrophic forgetting and overfitting problems at a time. Existing studies on lifelong GANs have proposed modulation-based methods to prevent catastrophic forgetting. However, they require considerable additional parameters and cannot generate high-fidelity and diverse images from limited data. On the other hand, the existing few-shot GANs suffer from severe catastrophic forgetting when learning multiple tasks. To alleviate these issues, we propose a framework called Lifelong Few-Shot GAN (LFS-GAN) that can generate high-quality and diverse images in lifelong few-shot image generation task. Our proposed framework learns each task using an efficient task-specific modulator - Learnable Factorized Tensor (LeFT). LeFT is rank-constrained and has a rich representation ability due to its unique reconstruction technique. Furthermore, we propose a novel mode seeking loss to improve the diversity of our model in low-data circumstances. Extensive experiments demonstrate that the proposed LFS-GAN can generate high-fidelity and diverse images without any forgetting and mode collapse in various domains, achieving state-of-the-art in lifelong few-shot image generation task. Surprisingly, we find that our LFS-GAN even outperforms the existing few-shot GANs in the few-shot image generation task. The code is available at Github.