One-shot Generative Domain Adaptation in 3D GANs

TL;DR

This paper introduces 3D-Adapter, a novel one-shot 3D generative domain adaptation method that fine-tunes pre-trained 3D GANs using a single reference image, achieving high fidelity, diversity, and consistency.

Contribution

The paper presents the first one-shot 3D GDA approach, combining selective fine-tuning, advanced loss functions, and progressive strategies for diverse and faithful 3D-aware image generation.

Findings

Achieves high-quality 3D domain adaptation with a single reference image.

Maintains multi-view and cross-domain consistency in generated images.

Extends to zero-shot scenarios and supports latent space tasks.

Abstract

3D-aware image generation necessitates extensive training data to ensure stable training and mitigate the risk of overfitting. This paper first considers a novel task known as One-shot 3D Generative Domain Adaptation (GDA), aimed at transferring a pre-trained 3D generator from one domain to a new one, relying solely on a single reference image. One-shot 3D GDA is characterized by the pursuit of specific attributes, namely, high fidelity, large diversity, cross-domain consistency, and multi-view consistency. Within this paper, we introduce 3D-Adapter, the first one-shot 3D GDA method, for diverse and faithful generation. Our approach begins by judiciously selecting a restricted weight set for fine-tuning, and subsequently leverages four advanced loss functions to facilitate adaptation. An efficient progressive fine-tuning strategy is also implemented to enhance the adaptation process. The synergy of these three technological components empowers 3D-Adapter to achieve remarkable performance, substantiated both quantitatively and qualitatively, across all desired properties of 3D GDA. Furthermore, 3D-Adapter seamlessly extends its capabilities to zero-shot scenarios, and preserves the potential for crucial tasks such as interpolation, reconstruction, and editing within the latent space of the pre-trained generator. Code will be available at https://github.com/iceli1007/3D-Adapter.