Few-shot Image Generation with Elastic Weight Consolidation

TL;DR

This paper introduces a method for few-shot image generation that adapts a pretrained model to new domains with minimal data by regularizing weight changes to preserve source diversity.

Contribution

It proposes a novel regularization technique during model adaptation that maintains source domain information while fitting limited target data, without adding extra parameters.

Findings

Effective generation with fewer than 10 examples

Preserves source diversity during adaptation

Performance depends on source-target dissimilarity

Abstract

Few-shot image generation seeks to generate more data of a given domain, with only few available training examples. As it is unreasonable to expect to fully infer the distribution from just a few observations (e.g., emojis), we seek to leverage a large, related source domain as pretraining (e.g., human faces). Thus, we wish to preserve the diversity of the source domain, while adapting to the appearance of the target. We adapt a pretrained model, without introducing any additional parameters, to the few examples of the target domain. Crucially, we regularize the changes of the weights during this adaptation, in order to best preserve the information of the source dataset, while fitting the target. We demonstrate the effectiveness of our algorithm by generating high-quality results of different target domains, including those with extremely few examples (e.g., <10). We also analyze the performance of our method with respect to some important factors, such as the number of examples and the dissimilarity between the source and target domain.