Synthetic History: Evaluating Visual Representations of the Past in Diffusion Models

TL;DR

This paper introduces a benchmark to evaluate how well text-to-image diffusion models depict historical contexts, revealing systematic inaccuracies and stereotypes in generated images across different eras.

Contribution

It presents a new benchmark dataset and evaluation protocol for assessing historical accuracy in diffusion model imagery, addressing a previously underexplored area.

Findings

Models often stereotype historical eras with unstated stylistic cues.

Generated images frequently contain anachronisms like modern artifacts.

Demographic representations in images often do not match historical plausibility.

Abstract

As Text-to-Image (TTI) diffusion models become increasingly influential in content creation, growing attention is being directed toward their societal and cultural implications. While prior research has primarily examined demographic and cultural biases, the ability of these models to accurately represent historical contexts remains largely underexplored. To address this gap, we introduce a benchmark for evaluating how TTI models depict historical contexts. The benchmark combines HistVis, a dataset of 30,000 synthetic images generated by three state-of-the-art diffusion models from carefully designed prompts covering universal human activities across multiple historical periods, with a reproducible evaluation protocol. We evaluate generated imagery across three key aspects: (1) Implicit Stylistic Associations: examining default visual styles associated with specific eras; (2) Historical Consistency: identifying anachronisms such as modern artifacts in pre-modern contexts; and (3) Demographic Representation: comparing generated racial and gender distributions against historically plausible baselines. Our findings reveal systematic inaccuracies in historically themed generated imagery, as TTI models frequently stereotype past eras by incorporating unstated stylistic cues, introduce anachronisms, and fail to reflect plausible demographic patterns. By providing a reproducible benchmark for historical representation in generated imagery, this work provides an initial step toward building more historically accurate TTI models.