Visualising the Attractor Landscape of Neural Cellular Automata

TL;DR

This paper explores techniques like manifold learning and topological data analysis to interpret the behavior of Neural Cellular Automata, revealing insights at both macro and micro levels.

Contribution

It applies novel interpretability methods to understand the learned behaviors of NCAs, bridging the gap between emergent behavior and comprehension.

Findings

Macroscopic analysis reveals simple underlying manifolds.

Microscopic analysis shows complex, high-dimensional manifolds.

Different techniques are needed for macro vs micro level understanding.

Abstract

As Neural Cellular Automata (NCAs) are increasingly applied outside of the toy models in Artificial Life, there is a pressing need to understand how they behave and to build appropriate routes to interpret what they have learnt. By their very nature, the benefits of training NCAs are balanced with a lack of interpretability: we can engineer emergent behaviour, but have limited ability to understand what has been learnt. In this paper, we apply a variety of techniques to pry open the NCA black box and glean some understanding of what it has learnt to do. We apply techniques from manifold learning (principal components analysis and both dense and sparse autoencoders) along with techniques from topological data analysis (persistent homology) to capture the NCA's underlying behavioural manifold, with varying success. Results show that when analysis is performed at a macroscopic level (i.e. taking the entire NCA state as a single data point), the underlying manifold is often quite simple and can be captured and analysed quite well. When analysis is performed at a microscopic level (i.e. taking the state of individual cells as a single data point), the manifold is highly complex and more complicated techniques are required in order to make sense of it.