Visual Anagrams: Generating Multi-View Optical Illusions with Diffusion Models

TL;DR

This paper introduces a zero-shot method using diffusion models to generate multi-view optical illusions, including visual anagrams, by estimating and combining noise from different views during the reverse diffusion process.

Contribution

The authors propose a novel zero-shot approach leveraging diffusion models to synthesize multi-view optical illusions, extending to complex pixel rearrangements like jigsaw puzzles.

Findings

Effective generation of multi-view illusions including rotations and flips

Method works for complex pixel permutations like jigsaw rearrangements

Extensible to multiple views beyond two

Abstract

We address the problem of synthesizing multi-view optical illusions: images that change appearance upon a transformation, such as a flip or rotation. We propose a simple, zero-shot method for obtaining these illusions from off-the-shelf text-to-image diffusion models. During the reverse diffusion process, we estimate the noise from different views of a noisy image, and then combine these noise estimates together and denoise the image. A theoretical analysis suggests that this method works precisely for views that can be written as orthogonal transformations, of which permutations are a subset. This leads to the idea of a visual anagram--an image that changes appearance under some rearrangement of pixels. This includes rotations and flips, but also more exotic pixel permutations such as a jigsaw rearrangement. Our approach also naturally extends to illusions with more than two views. We provide both qualitative and quantitative results demonstrating the effectiveness and flexibility of our method. Please see our project webpage for additional visualizations and results: https://dangeng.github.io/visual_anagrams/