Aligned Stable Inpainting: Mitigating Unwanted Object Insertion and Preserving Color Consistency
Yikai Wang, Junqiu Yu, Chenjie Cao, Xiangyang Xue, Yanwei Fu

TL;DR
This paper introduces ASUKA, a framework that enhances generative image inpainting by reducing unwanted object hallucination and color inconsistency through reconstruction priors and a specialized VAE decoder, improving realism and color fidelity.
Contribution
The paper proposes a novel post-hoc framework, ASUKA, that significantly mitigates hallucinated objects and color shifts in pre-trained inpainting models without retraining them.
Findings
ASUKA suppresses unwanted object hallucination effectively.
ASUKA improves color consistency in inpainted images.
Outperforms existing inpainting methods on benchmark datasets.
Abstract
Generative image inpainting can produce realistic, high-fidelity results even with large, irregular masks. However, existing methods still face key issues that make inpainted images look unnatural. In this paper, we identify two main problems: (1) Unwanted object insertion: generative models may hallucinate arbitrary objects in the masked region that do not match the surrounding context. (2) Color inconsistency: inpainted regions often exhibit noticeable color shifts, leading to smeared textures and degraded image quality. We analyze the underlying causes of these issues and propose efficient post-hoc solutions for pre-trained inpainting models. Specifically, we introduce the principled framework of Aligned Stable inpainting with UnKnown Areas prior (ASUKA). To reduce unwanted object insertion, we use reconstruction-based priors to guide the generative model, suppressing hallucinated…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Image Enhancement Techniques · Computer Graphics and Visualization Techniques
