You Don't Have to Be Perfect to Be Amazing: Unveil the Utility of Synthetic Images
Xiaodan Xing, Federico Felder, Yang Nan, Giorgos Papanastasiou, Walsh, Simon, Guang Yang

TL;DR
This paper evaluates synthetic images from deep generative models across multiple metrics, revealing trade-offs between fidelity, variety, privacy, and utility, and demonstrating high utility can be achieved even with low fidelity or variety images.
Contribution
It introduces a comprehensive evaluation framework for synthetic images, analyzing over 100k chest X-rays to uncover relationships between quality, utility, and privacy, and provides insights for utility-aware generative models.
Findings
Trade-off exists between fidelity, variety, and privacy of synthetic images.
High utility can be achieved with images of low fidelity or variety.
High utility and privacy can coexist in synthetic images.
Abstract
Synthetic images generated from deep generative models have the potential to address data scarcity and data privacy issues. The selection of synthesis models is mostly based on image quality measurements, and most researchers favor synthetic images that produce realistic images, i.e., images with good fidelity scores, such as low Fr\'echet Inception Distance (FID) and high Peak Signal-To-Noise Ratio (PSNR). However, the quality of synthetic images is not limited to fidelity, and a wide spectrum of metrics should be evaluated to comprehensively measure the quality of synthetic images. In addition, quality metrics are not truthful predictors of the utility of synthetic images, and the relations between these evaluation metrics are not yet clear. In this work, we have established a comprehensive set of evaluators for synthetic images, including fidelity, variety, privacy, and utility. By…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · AI in cancer detection
