Variational Neural Cellular Automata

TL;DR

The paper introduces Variational Neural Cellular Automata (VNCA), a probabilistic generative model inspired by biological cellular growth, capable of learning diverse, self-organizing data distributions and stable attractors despite its simplicity.

Contribution

It presents VNCA as a novel probabilistic cellular automaton model that learns to generate diverse data and recover from damage, differing from previous deterministic approaches.

Findings

VNCA reconstructs samples effectively.

It generates diverse outputs from a shared vector.

It learns stable attractors that recover from damage.

Abstract

In nature, the process of cellular growth and differentiation has lead to an amazing diversity of organisms -- algae, starfish, giant sequoia, tardigrades, and orcas are all created by the same generative process. Inspired by the incredible diversity of this biological generative process, we propose a generative model, the Variational Neural Cellular Automata (VNCA), which is loosely inspired by the biological processes of cellular growth and differentiation. Unlike previous related works, the VNCA is a proper probabilistic generative model, and we evaluate it according to best practices. We find that the VNCA learns to reconstruct samples well and that despite its relatively few parameters and simple local-only communication, the VNCA can learn to generate a large variety of output from information encoded in a common vector format. While there is a significant gap to the current state-of-the-art in terms of generative modeling performance, we show that the VNCA can learn a purely self-organizing generative process of data. Additionally, we show that the VNCA can learn a distribution of stable attractors that can recover from significant damage.