
TL;DR
This paper explores the challenge of quantifying the realism of generated data, proposing the concept of a universal critic inspired by algorithmic information theory to better evaluate and understand realism in generative models.
Contribution
It introduces the notion of a universal critic that does not require adversarial training, providing a new theoretical framework for assessing data realism.
Findings
Universal critics serve as a theoretical guide for realism evaluation.
Good generative models are insufficient without proper realism assessment.
The paper offers insights into the limitations of current evaluation methods.
Abstract
The last decade has seen tremendous progress in our ability to generate realistic-looking data, be it images, text, audio, or video. Here, we discuss the closely related problem of quantifying realism, that is, designing functions that can reliably tell realistic data from unrealistic data. This problem turns out to be significantly harder to solve and remains poorly understood, despite its prevalence in machine learning and recent breakthroughs in generative AI. Drawing on insights from algorithmic information theory, we discuss why this problem is challenging, why a good generative model alone is insufficient to solve it, and what a good solution would look like. In particular, we introduce the notion of a universal critic, which unlike adversarial critics does not require adversarial training. While universal critics are not immediately practical, they can serve both as a North Star…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAesthetic Perception and Analysis · Visual Culture and Art Theory
