Geometry Preserving Loss Functions Promote Improved Adaptation of Blackbox Generative Model

TL;DR

This paper introduces a geometry-preserving loss function for domain adaptation of pre-trained GANs, improving their ability to generate samples from target distributions under real distribution shifts.

Contribution

The authors propose a novel end-to-end pipeline that leverages geometry-preserving loss functions and improved GAN inversion for better domain adaptation of large-scale generative models.

Findings

Geometry-preserving loss improves GAN adaptation to target distributions.

The method outperforms traditional loss functions in experiments with StyleGANs.

Enhanced latent space representations lead to more accurate target distribution sampling.

Abstract

Adaptation of blackbox generative models has been widely studied recently through the exploration of several methods including generator fine-tuning, latent space searches, leveraging singular value decomposition, and so on. However, adapting large-scale generative AI tools to specific use cases continues to be challenging, as many of these industry-grade models are not made widely available. The traditional approach of fine-tuning certain layers of a generative network is not feasible due to the expense of storing and fine-tuning generative models, as well as the restricted access to weights and gradients. Recognizing these challenges, we propose a novel end-to-end pipeline aimed at domain adaptation by leveraging geometry-preserving loss functions in conjunction to pre-trained generative adversarial networks (GANs). Our method rethinks the problem of adaptation by re-contextualizing the role of GAN inversion in obtaining accurate latent space representations. Extending the ability of existing state-of-the-art inverters, we preserve pair-wise distances between tangent spaces to successfully train a latent generative model to produce samples from the target distribution. We evaluate our proposed pipeline on StyleGANs with real distribution shifts and demonstrate that the introduction of the geometry preserving loss function lends to improved adaptation of generative models compared to other traditional loss functions.