Fractal Generative Models

TL;DR

This paper introduces fractal generative models that recursively compose atomic modules into self-similar architectures, demonstrating promising results in image generation tasks and opening new avenues in generative modeling.

Contribution

It proposes a novel fractal framework for generative models, enabling recursive self-similar architectures with demonstrated effectiveness in image generation.

Findings

Strong likelihood estimation performance

High-quality pixel-by-pixel image generation

Potential for new generative modeling paradigms

Abstract

Modularization is a cornerstone of computer science, abstracting complex functions into atomic building blocks. In this paper, we introduce a new level of modularization by abstracting generative models into atomic generative modules. Analogous to fractals in mathematics, our method constructs a new type of generative model by recursively invoking atomic generative modules, resulting in self-similar fractal architectures that we call fractal generative models. As a running example, we instantiate our fractal framework using autoregressive models as the atomic generative modules and examine it on the challenging task of pixel-by-pixel image generation, demonstrating strong performance in both likelihood estimation and generation quality. We hope this work could open a new paradigm in generative modeling and provide a fertile ground for future research. Code is available at https://github.com/LTH14/fractalgen.