Schr\"{o}dinger's Bat: Diffusion Models Sometimes Generate Polysemous Words in Superposition

TL;DR

This paper investigates why diffusion models sometimes generate images with multiple meanings of a word, revealing that encodings of polysemous words are stored as superpositions, which can lead to images representing multiple senses simultaneously.

Contribution

It demonstrates that polysemous words are encoded as superpositions in CLIP, and that diffusion models produce images reflecting these superpositions, explaining the homonym duplication phenomenon.

Findings

Diffusion models can generate images with multiple word senses from summed encodings.

CLIP encodes polysemous words as superpositions of meanings.

Linear algebra techniques can manipulate these superpositions to influence generated images.

Abstract

Recent work has shown that despite their impressive capabilities, text-to-image diffusion models such as DALL-E 2 (Ramesh et al., 2022) can display strange behaviours when a prompt contains a word with multiple possible meanings, often generating images containing both senses of the word (Rassin et al., 2022). In this work we seek to put forward a possible explanation of this phenomenon. Using the similar Stable Diffusion model (Rombach et al., 2022), we first show that when given an input that is the sum of encodings of two distinct words, the model can produce an image containing both concepts represented in the sum. We then demonstrate that the CLIP encoder used to encode prompts (Radford et al., 2021) encodes polysemous words as a superposition of meanings, and that using linear algebraic techniques we can edit these representations to influence the senses represented in the generated images. Combining these two findings, we suggest that the homonym duplication phenomenon described by Rassin et al. (2022) is caused by diffusion models producing images representing both of the meanings that are present in superposition in the encoding of a polysemous word.