Domain Expansion of Image Generators

TL;DR

This paper introduces a method to expand pretrained image generators to include new related domains by minimally perturbing the latent space, enabling multi-domain modeling without increasing model size.

Contribution

The authors propose a novel domain expansion technique that leverages dormant latent directions to incorporate new domains into existing generators without retraining or enlarging the model.

Findings

Pretrained generators contain unused latent directions for new domains.

The method allows adding hundreds of new domains to a single generator.

Expanded models support smooth domain transitions and domain composition.

Abstract

Can one inject new concepts into an already trained generative model, while respecting its existing structure and knowledge? We propose a new task - domain expansion - to address this. Given a pretrained generator and novel (but related) domains, we expand the generator to jointly model all domains, old and new, harmoniously. First, we note the generator contains a meaningful, pretrained latent space. Is it possible to minimally perturb this hard-earned representation, while maximally representing the new domains? Interestingly, we find that the latent space offers unused, "dormant" directions, which do not affect the output. This provides an opportunity: By "repurposing" these directions, we can represent new domains without perturbing the original representation. In fact, we find that pretrained generators have the capacity to add several - even hundreds - of new domains! Using our expansion method, one "expanded" model can supersede numerous domain-specific models, without expanding the model size. Additionally, a single expanded generator natively supports smooth transitions between domains, as well as composition of domains. Code and project page available at https://yotamnitzan.github.io/domain-expansion/.