What Do AI-Generated Images Want?

TL;DR

This paper explores the agency of AI-generated images, arguing they desire specificity and concreteness due to their abstract nature, and critiques the misconception that text and images are directly interchangeable in multimodal models.

Contribution

It reframes Mitchell's question in the context of AI image generation, highlighting the abstract nature of AI images and critiquing the assumptions about text-image interchangeability.

Findings

AI images are fundamentally abstract and desire concreteness.

Multimodal models obscure the representational differences between text and images.

The pipeline from text to image creates an illusion of direct transformation.

Abstract

W.J.T. Mitchell's influential essay 'What do pictures want?' shifts the theoretical focus away from the interpretative act of understanding pictures and from the motivations of the humans who create them to the possibility that the picture itself is an entity with agency and wants. In this article, I reframe Mitchell's question in light of contemporary AI image generation tools to ask: what do AI-generated images want? Drawing from art historical discourse on the nature of abstraction, I argue that AI-generated images want specificity and concreteness because they are fundamentally abstract. Multimodal text-to-image models, which are the primary subject of this article, are based on the premise that text and image are interchangeable or exchangeable tokens and that there is a commensurability between them, at least as represented mathematically in data. The user pipeline that sees textual input become visual output, however, obscures this representational regress and makes it seem like one form transforms into the other -- as if by magic.