CR-Fill: Generative Image Inpainting with Auxiliary Contexutal Reconstruction
Yu Zeng, Zhe Lin, Huchuan Lu, Vishal M. Patel

TL;DR
CR-Fill introduces an attention-free generative inpainting method trained with an auxiliary contextual reconstruction task, improving plausibility and reducing artifacts without increasing inference complexity.
Contribution
The paper proposes a novel attention-free inpainting approach using a joint training scheme with a CR loss, eliminating the need for attention during inference.
Findings
Outperforms state-of-the-art methods quantitatively.
Produces more visually plausible inpainted images.
Reduces computational overhead during inference.
Abstract
Recent deep generative inpainting methods use attention layers to allow the generator to explicitly borrow feature patches from the known region to complete a missing region. Due to the lack of supervision signals for the correspondence between missing regions and known regions, it may fail to find proper reference features, which often leads to artifacts in the results. Also, it computes pair-wise similarity across the entire feature map during inference bringing a significant computational overhead. To address this issue, we propose to teach such patch-borrowing behavior to an attention-free generator by joint training of an auxiliary contextual reconstruction task, which encourages the generated output to be plausible even when reconstructed by surrounding regions. The auxiliary branch can be seen as a learnable loss function, i.e. named as contextual reconstruction (CR) loss, where…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Advanced Image Processing Techniques · Computer Graphics and Visualization Techniques
MethodsInpainting
