Towards Self-Assembling Artificial Neural Networks through Neural Developmental Programs

TL;DR

This paper explores a novel approach to neural network design inspired by biological development, where networks grow through neural developmental programs guided by local communication, aiming for self-assembling architectures.

Contribution

It introduces the concept of neural developmental programs that enable neural networks to self-organize, mimicking biological growth processes, and investigates their application across various learning paradigms.

Findings

Neural growth impacts performance on multiple benchmarks.

Developmental processes can be guided by neural networks using local communication.

Self-organization offers new avenues for neural network design.

Abstract

Biological nervous systems are created in a fundamentally different way than current artificial neural networks. Despite its impressive results in a variety of different domains, deep learning often requires considerable engineering effort to design high-performing neural architectures. By contrast, biological nervous systems are grown through a dynamic self-organizing process. In this paper, we take initial steps toward neural networks that grow through a developmental process that mirrors key properties of embryonic development in biological organisms. The growth process is guided by another neural network, which we call a Neural Developmental Program (NDP) and which operates through local communication alone. We investigate the role of neural growth on different machine learning benchmarks and different optimization methods (evolutionary training, online RL, offline RL, and supervised learning). Additionally, we highlight future research directions and opportunities enabled by having self-organization driving the growth of neural networks.