Borrowing from anything: A generalizable framework for reference-guided instance editing
Shengxiao Zhou, Chenghua Li, Jianhao Huang, Qinghao Hu, Yifan Zhang

TL;DR
GENIE is a novel framework for reference-guided instance editing that explicitly disentangles intrinsic appearance from extrinsic attributes, enabling more accurate and robust editing by correcting spatial misalignments and selectively borrowing features.
Contribution
The paper introduces GENIE, a generalizable framework with modules for spatial alignment, adaptive residual scaling, and progressive attention fusion, advancing the state-of-the-art in disentanglement-based instance editing.
Findings
Achieves state-of-the-art fidelity on the AnyInsertion dataset.
Demonstrates robustness and generalizability across diverse editing scenarios.
Effectively disentangles intrinsic and extrinsic information for precise editing.
Abstract
Reference-guided instance editing is fundamentally limited by semantic entanglement, where a reference's intrinsic appearance is intertwined with its extrinsic attributes. The key challenge lies in disentangling what information should be borrowed from the reference, and determining how to apply it appropriately to the target. To tackle this challenge, we propose GENIE, a Generalizable Instance Editing framework capable of achieving explicit disentanglement. GENIE first corrects spatial misalignments with a Spatial Alignment Module (SAM). Then, an Adaptive Residual Scaling Module (ARSM) learns what to borrow by amplifying salient intrinsic cues while suppressing extrinsic attributes, while a Progressive Attention Fusion (PAF) mechanism learns how to render this appearance onto the target, preserving its structure. Extensive experiments on the challenging AnyInsertion dataset demonstrate…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMachine Learning and Data Classification · Handwritten Text Recognition Techniques · Cell Image Analysis Techniques
