PasteGAN: A Semi-Parametric Method to Generate Image from Scene Graph
Yikang Li, Tao Ma, Yeqi Bai, Nan Duan, Sining Wei, Xiaogang Wang

TL;DR
PasteGAN is a semi-parametric approach that generates controllable, high-quality images from scene graphs and object crops, allowing detailed manipulation of object appearances and interactions.
Contribution
The paper introduces PasteGAN, a novel semi-parametric method that combines scene graphs and object crops for more controllable and detailed image synthesis.
Findings
Outperforms SOTA on Inception Score, Diversity Score, FID
Generates complex and diverse images with specified objects
Effectively integrates object crops and scene graph relationships
Abstract
Despite some exciting progress on high-quality image generation from structured(scene graphs) or free-form(sentences) descriptions, most of them only guarantee the image-level semantical consistency, i.e. the generated image matching the semantic meaning of the description. They still lack the investigations on synthesizing the images in a more controllable way, like finely manipulating the visual appearance of every object. Therefore, to generate the images with preferred objects and rich interactions, we propose a semi-parametric method, PasteGAN, for generating the image from the scene graph and the image crops, where spatial arrangements of the objects and their pair-wise relationships are defined by the scene graph and the object appearances are determined by the given object crops. To enhance the interactions of the objects in the output, we design a Crop Refining Network and an…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Advanced Image and Video Retrieval Techniques · Generative Adversarial Networks and Image Synthesis
