Meta Internal Learning

TL;DR

This paper introduces a meta-learning framework for internal image generation that trains a hypernetwork to produce single-image GANs, enabling faster training, interpolation, and improved modeling of internal image statistics.

Contribution

It proposes a meta-learning approach that generates single-image GANs from a collection of images, enhancing efficiency and capabilities over traditional methods.

Findings

Models are as effective as single-image GANs for image tasks.

Significantly reduces training time per image.

Enables interpolation and feedforward generation of images.

Abstract

Internal learning for single-image generation is a framework, where a generator is trained to produce novel images based on a single image. Since these models are trained on a single image, they are limited in their scale and application. To overcome these issues, we propose a meta-learning approach that enables training over a collection of images, in order to model the internal statistics of the sample image more effectively. In the presented meta-learning approach, a single-image GAN model is generated given an input image, via a convolutional feedforward hypernetwork $f$. This network is trained over a dataset of images, allowing for feature sharing among different models, and for interpolation in the space of generative models. The generated single-image model contains a hierarchy of multiple generators and discriminators. It is therefore required to train the meta-learner in an adversarial manner, which requires careful design choices that we justify by a theoretical analysis. Our results show that the models obtained are as suitable as single-image GANs for many common image applications, significantly reduce the training time per image without loss in performance, and introduce novel capabilities, such as interpolation and feedforward modeling of novel images.