PIXELS: Progressive Image Xemplar-based Editing with Latent Surgery
Shristi Das Biswas, Matthew Shreve, Xuelu Li, Prateek Singhal, Kaushik, Roy

TL;DR
PIXELS is a novel framework that enables fine-grained, exemplar-based image editing using off-the-shelf diffusion models, allowing users to make localized and progressive modifications without retraining or fine-tuning models.
Contribution
The paper introduces PIXELS, a new inference-only method for exemplar-driven image editing that offers granular control and progressive updates, overcoming limitations of prior techniques.
Findings
Achieves high-quality, localized edits with improved quantitative metrics.
Enables progressive, region-specific modifications without retraining.
Demonstrates effectiveness across diverse base models and exemplar inputs.
Abstract
Recent advancements in language-guided diffusion models for image editing are often bottle-necked by cumbersome prompt engineering to precisely articulate desired changes. An intuitive alternative calls on guidance from in-the-wild image exemplars to help users bring their imagined edits to life. Contemporary exemplar-based editing methods shy away from leveraging the rich latent space learnt by pre-existing large text-to-image (TTI) models and fall back on training with curated objective functions to achieve the task. Though somewhat effective, this demands significant computational resources and lacks compatibility with diverse base models and arbitrary exemplar count. On further investigation, we also find that these techniques restrict user control to only applying uniform global changes over the entire edited region. In this paper, we introduce a novel framework for progressive…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsAdvanced Image and Video Retrieval Techniques · Generative Adversarial Networks and Image Synthesis · Multimodal Machine Learning Applications
MethodsDiffusion · Balanced Selection
