PICS: Pairwise Image Compositing with Spatial Interactions

TL;DR

PICS introduces a self-supervised, interaction-aware image compositing method that models spatial relations and object interactions to improve the quality and stability of pairwise image edits, outperforming existing approaches.

Contribution

The paper proposes a novel Interaction Transformer with mask-guided Mixture-of-Experts and adaptive blending for improved spatial coherence in image compositing.

Findings

Superior compositing quality over state-of-the-art methods

Enhanced stability in pairwise image edits

Effective handling of geometric variations

Abstract

Despite strong single-turn performance, diffusion-based image compositing often struggles to preserve coherent spatial relations in pairwise or sequential edits, where subsequent insertions may overwrite previously generated content and disrupt physical consistency. We introduce PICS, a self-supervised composition-by-decomposition paradigm that composes objects in parallel while explicitly modeling the compositional interactions among (fully-/partially-)visible objects and background. At its core, an Interaction Transformer employs mask-guided Mixture-of-Experts to route background, exclusive, and overlap regions to dedicated experts, with an adaptive {\alpha}-blending strategy that infers a compatibility-aware fusion of overlapping objects while preserving boundary fidelity. To further enhance robustness to geometric variations, we incorporate geometry-aware augmentations covering both out-of-plane and in-plane pose changes of objects. Our method delivers superior pairwise compositing quality and substantially improved stability, with extensive evaluations across virtual try-on, indoor, and street scene settings showing consistent gains over state-of-the-art baselines. Code and data are available at https://github.com/RyanHangZhou/PICS