Geometric Image Synthesis
Hassan Abu Alhaija, Siva Karthik Mustikovela, Andreas Geiger, Carsten, Rother

TL;DR
This paper introduces a geometry-aware neural network framework for generating realistic images from 3D scene information, improving structural consistency and generalization over previous methods.
Contribution
The proposed GIS framework integrates scene geometry and segmentation into neural image synthesis, enhancing realism, control, and generalization in generated images.
Findings
Successfully inserts vehicles into outdoor scenes.
Generates novel views of objects beyond training data.
Improves training data for segmentation models.
Abstract
The task of generating natural images from 3D scenes has been a long standing goal in computer graphics. On the other hand, recent developments in deep neural networks allow for trainable models that can produce natural-looking images with little or no knowledge about the scene structure. While the generated images often consist of realistic looking local patterns, the overall structure of the generated images is often inconsistent. In this work we propose a trainable, geometry-aware image generation method that leverages various types of scene information, including geometry and segmentation, to create realistic looking natural images that match the desired scene structure. Our geometrically-consistent image synthesis method is a deep neural network, called Geometry to Image Synthesis (GIS) framework, which retains the advantages of a trainable method, e.g., differentiability and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsComputer Graphics and Visualization Techniques · Advanced Vision and Imaging · Generative Adversarial Networks and Image Synthesis
