GIRAFFE HD: A High-Resolution 3D-aware Generative Model

TL;DR

GIRAFFE HD is a high-resolution 3D-aware generative model that produces controllable, high-quality images at 512x512 resolution and above by leveraging style-based rendering and foreground-background disentanglement.

Contribution

It extends GIRAFFE to high-resolution image generation using style-based rendering and disentangled foreground-background synthesis.

Findings

Achieves state-of-the-art high-resolution 3D controllable image generation

Generates images at 512x512 resolution and above

Demonstrates effectiveness on multiple natural image datasets

Abstract

3D-aware generative models have shown that the introduction of 3D information can lead to more controllable image generation. In particular, the current state-of-the-art model GIRAFFE can control each object's rotation, translation, scale, and scene camera pose without corresponding supervision. However, GIRAFFE only operates well when the image resolution is low. We propose GIRAFFE HD, a high-resolution 3D-aware generative model that inherits all of GIRAFFE's controllable features while generating high-quality, high-resolution images ($512^2$ resolution and above). The key idea is to leverage a style-based neural renderer, and to independently generate the foreground and background to force their disentanglement while imposing consistency constraints to stitch them together to composite a coherent final image. We demonstrate state-of-the-art 3D controllable high-resolution image generation on multiple natural image datasets.