Texture Generation with Neural Cellular Automata

TL;DR

This paper introduces a neural cellular automata-based method for texture generation that learns from a single image, producing high-fidelity, parallelizable textures with robustness and interesting dynamic properties.

Contribution

It demonstrates a novel approach to texture synthesis using NCA trained on a single template, highlighting its efficiency, local rule learning, and robustness over existing methods.

Findings

High-fidelity texture generation from a single image

Fast convergence and parallelizable synthesis process

Robustness to damage and non-stationary dynamics

Abstract

Neural Cellular Automata (NCA) have shown a remarkable ability to learn the required rules to "grow" images, classify morphologies, segment images, as well as to do general computation such as path-finding. We believe the inductive prior they introduce lends itself to the generation of textures. Textures in the natural world are often generated by variants of locally interacting reaction-diffusion systems. Human-made textures are likewise often generated in a local manner (textile weaving, for instance) or using rules with local dependencies (regular grids or geometric patterns). We demonstrate learning a texture generator from a single template image, with the generation method being embarrassingly parallel, exhibiting quick convergence and high fidelity of output, and requiring only some minimal assumptions around the underlying state manifold. Furthermore, we investigate properties of the learned models that are both useful and interesting, such as non-stationary dynamics and an inherent robustness to damage. Finally, we make qualitative claims that the behaviour exhibited by the NCA model is a learned, distributed, local algorithm to generate a texture, setting our method apart from existing work on texture generation. We discuss the advantages of such a paradigm.