Person-In-Situ: Scene-Consistent Human Image Insertion with Occlusion-Aware Pose Control
Shun Masuda, Yuki Endo, Yoshihiro Kanamori

TL;DR
This paper introduces two novel methods for inserting human figures into scene images with accurate pose control and occlusion handling, improving scene consistency and realism without needing explicit occlusion masks.
Contribution
The paper presents two new approaches that enable scene-consistent human image insertion with explicit pose control and occlusion awareness, advancing beyond prior methods.
Findings
Both methods outperform existing approaches in scene consistency.
They accurately handle occlusions without explicit masks.
They allow user-specified pose control.
Abstract
Compositing human figures into scene images has broad applications in areas such as entertainment and advertising. However, existing methods often cannot handle occlusion of the inserted person by foreground objects and unnaturally place the person in the frontmost layer. Moreover, they offer limited control over the inserted person's pose. To address these challenges, we propose two methods. Both allow explicit pose control via a 3D body model and leverage latent diffusion models to synthesize the person at a contextually appropriate depth, naturally handling occlusions without requiring occlusion masks. The first is a two-stage approach: the model first learns a depth map of the scene with the person through supervised learning, and then synthesizes the person accordingly. The second method learns occlusion implicitly and synthesizes the person directly from input data without…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Human Pose and Action Recognition · Advanced Vision and Imaging
MethodsDiffusion
