Smoothness Similarity Regularization for Few-Shot GAN Adaptation

TL;DR

This paper introduces a smoothness similarity regularization technique that enhances few-shot GAN adaptation, especially when source and target domains differ significantly, by transferring learned smoothness properties to improve stability and performance.

Contribution

The paper proposes a novel regularization method that effectively transfers smoothness from pre-trained GANs to diverse few-shot target domains, addressing structural dissimilarity issues.

Findings

Outperforms prior methods on dissimilar domains

Matches state-of-the-art on similar domains

Improves training stability and reduces memorization

Abstract

The task of few-shot GAN adaptation aims to adapt a pre-trained GAN model to a small dataset with very few training images. While existing methods perform well when the dataset for pre-training is structurally similar to the target dataset, the approaches suffer from training instabilities or memorization issues when the objects in the two domains have a very different structure. To mitigate this limitation, we propose a new smoothness similarity regularization that transfers the inherently learned smoothness of the pre-trained GAN to the few-shot target domain even if the two domains are very different. We evaluate our approach by adapting an unconditional and a class-conditional GAN to diverse few-shot target domains. Our proposed method significantly outperforms prior few-shot GAN adaptation methods in the challenging case of structurally dissimilar source-target domains, while performing on par with the state of the art for similar source-target domains.