Growing 3D Artefacts and Functional Machines with Neural Cellular Automata

TL;DR

This paper extends Neural Cellular Automata to 3D, demonstrating their ability to grow complex structures and functional machines in a Minecraft environment, with potential applications in morphogenetic simulations.

Contribution

It introduces a 3D NCA architecture using 3D convolutions and demonstrates their capacity to generate and regenerate complex 3D structures and machines.

Findings

NCAs can grow complex 3D structures like castles and trees

NCAs can regenerate parts of functional machines

The approach enables morphogenetic simulations in 3D environments

Abstract

Neural Cellular Automata (NCAs) have been proven effective in simulating morphogenetic processes, the continuous construction of complex structures from very few starting cells. Recent developments in NCAs lie in the 2D domain, namely reconstructing target images from a single pixel or infinitely growing 2D textures. In this work, we propose an extension of NCAs to 3D, utilizing 3D convolutions in the proposed neural network architecture. Minecraft is selected as the environment for our automaton since it allows the generation of both static structures and moving machines. We show that despite their simplicity, NCAs are capable of growing complex entities such as castles, apartment blocks, and trees, some of which are composed of over 3,000 blocks. Additionally, when trained for regeneration, the system is able to regrow parts of simple functional machines, significantly expanding the capabilities of simulated morphogenetic systems. The code for the experiment in this paper can be found at: https://github.com/real-itu/3d-artefacts-nca.