Person-in-Context Synthesiswith Compositional Structural Space
Weidong Yin, Ziwei Liu, Leonid Sigal

TL;DR
This paper introduces a novel framework for synthesizing complex images with multiple people in context, using a compositional structural space that encodes shape, location, and appearance, enabling more realistic and controllable image generation.
Contribution
The paper proposes a new approach with separate neural branches and a shared structural space for synthesizing multi-person images in context, addressing limitations of existing methods.
Findings
Outperforms state-of-the-art in synthesis quality on COCO-Stuff and Visual Genome datasets.
Effectively encodes shape, location, and appearance in a disentangled manner.
Demonstrates versatility in generating diverse, contextually consistent person images.
Abstract
Despite significant progress, controlled generation of complex images with interacting people remains difficult. Existing layout generation methods fall short of synthesizing realistic person instances; while pose-guided generation approaches focus on a single person and assume simple or known backgrounds. To tackle these limitations, we propose a new problem, \textbf{Persons in Context Synthesis}, which aims to synthesize diverse person instance(s) in consistent contexts, with user control over both. The context is specified by the bounding box object layout which lacks shape information, while pose of the person(s) by keypoints which are sparsely annotated. To handle the stark difference in input structures, we proposed two separate neural branches to attentively composite the respective (context/person) inputs into shared ``compositional structural space'', which encodes shape,…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Human Pose and Action Recognition · Multimodal Machine Learning Applications
