Neural Cellular Automata and Deep Equilibrium Models

TL;DR

This paper explores the relationship between Neural Cellular Automata and Deep Equilibrium Models, demonstrating their similarities through a simple model and discussing potential future integrations in deep learning.

Contribution

It provides a comparative analysis of NCA and DEQ, and trains a basic DEQ convolutional model to highlight their conceptual connections.

Findings

NCA and DEQ share underlying principles.

A simple DEQ convolutional model can emulate NCA behaviors.

Discussion on future hybrid approaches for deep learning.

Abstract

This essay discusses the connections and differences between two emerging paradigms in deep learning, namely Neural Cellular Automata and Deep Equilibrium Models, and train a simple Deep Equilibrium Convolutional model to demonstrate the inherent similarity of NCA and DEQ based methods. Finally, this essay speculates about ways to combine theoretical and practical aspects of both approaches for future research.