Improving Few-shot Image Generation by Structural Discrimination and Textural Modulation

TL;DR

This paper introduces a novel few-shot image generation model that uses structural discrimination and textural modulation to improve diversity, semantic fusion, and layout accuracy, achieving state-of-the-art results.

Contribution

The paper proposes TexMod for fine-grained semantic injection and StructD for layout guidance, enhancing existing few-shot generation methods.

Findings

Achieves state-of-the-art performance on three datasets.

Improves diversity and semantic coherence in generated images.

Techniques can be integrated into existing models for better results.

Abstract

Few-shot image generation, which aims to produce plausible and diverse images for one category given a few images from this category, has drawn extensive attention. Existing approaches either globally interpolate different images or fuse local representations with pre-defined coefficients. However, such an intuitive combination of images/features only exploits the most relevant information for generation, leading to poor diversity and coarse-grained semantic fusion. To remedy this, this paper proposes a novel textural modulation (TexMod) mechanism to inject external semantic signals into internal local representations. Parameterized by the feedback from the discriminator, our TexMod enables more fined-grained semantic injection while maintaining the synthesis fidelity. Moreover, a global structural discriminator (StructD) is developed to explicitly guide the model to generate images with reasonable layout and outline. Furthermore, the frequency awareness of the model is reinforced by encouraging the model to distinguish frequency signals. Together with these techniques, we build a novel and effective model for few-shot image generation. The effectiveness of our model is identified by extensive experiments on three popular datasets and various settings. Besides achieving state-of-the-art synthesis performance on these datasets, our proposed techniques could be seamlessly integrated into existing models for a further performance boost.