Perceptual Artifacts Localization for Image Synthesis Tasks
Lingzhi Zhang, Zhengjie Xu, Connelly Barnes, Yuqian Zhou, Qing Liu, He, Zhang, Sohrab Amirghodsi, Zhe Lin, Eli Shechtman, Jianbo Shi

TL;DR
This paper introduces a dataset and segmentation model for localizing perceptual artifacts in images generated by deep models, enabling automatic correction and quality assessment across various synthesis tasks.
Contribution
The study provides a new annotated dataset, a versatile segmentation model for artifact localization, and a novel inpainting pipeline for artifact rectification in image synthesis.
Findings
Effective artifact localization across multiple synthesis tasks
Model adapts to unseen models with minimal training data
Proposed pipeline improves image quality by correcting artifacts
Abstract
Recent advancements in deep generative models have facilitated the creation of photo-realistic images across various tasks. However, these generated images often exhibit perceptual artifacts in specific regions, necessitating manual correction. In this study, we present a comprehensive empirical examination of Perceptual Artifacts Localization (PAL) spanning diverse image synthesis endeavors. We introduce a novel dataset comprising 10,168 generated images, each annotated with per-pixel perceptual artifact labels across ten synthesis tasks. A segmentation model, trained on our proposed dataset, effectively localizes artifacts across a range of tasks. Additionally, we illustrate its proficiency in adapting to previously unseen models using minimal training samples. We further propose an innovative zoom-in inpainting pipeline that seamlessly rectifies perceptual artifacts in the generated…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Perceptual Artifacts Localization for Image Synthesis Tasks· youtube
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Cell Image Analysis Techniques · Advanced Vision and Imaging
MethodsInpainting
