Towards Source-Aware Object Swapping with Initial Noise Perturbation

TL;DR

This paper introduces SourceSwap, a self-supervised framework for object swapping that learns cross-object alignment using frequency-separated perturbations, enabling high-quality, zero-shot editing without per-object finetuning.

Contribution

It proposes a novel frequency-based pseudo pair synthesis method and a dual U-Net architecture for source-aware object swapping without requiring paired data or finetuning.

Findings

Outperforms existing methods in fidelity and scene preservation.

Enables zero-shot inference and diverse editing tasks.

Introduces SourceBench, a new high-quality benchmark.

Abstract

Object swapping aims to replace a source object in a scene with a reference object while preserving object fidelity, scene fidelity, and object-scene harmony. Existing methods either require per-object finetuning and slow inference or rely on extra paired data that mostly depict the same object across contexts, forcing models to rely on background cues rather than learning cross-object alignment. We propose SourceSwap, a self-supervised and source-aware framework that learns cross-object alignment. Our key insight is to synthesize high-quality pseudo pairs from any image via a frequency-separated perturbation in the initial-noise space, which alters appearance while preserving pose, coarse shape, and scene layout, requiring no videos, multi-view data, or additional images. We then train a dual U-Net with full-source conditioning and a noise-free reference encoder, enabling direct inter-object alignment, zero-shot inference without per-object finetuning, and lightweight iterative refinement. We further introduce SourceBench, a high-quality benchmark with higher resolution, more categories, and richer interactions. Experiments demonstrate that SourceSwap achieves superior fidelity, stronger scene preservation, and more natural harmony, and it transfers well to edits such as subject-driven refinement and face swapping.