Structurally Flexible Neural Networks: Evolving the Building Blocks for General Agents

TL;DR

This paper introduces a method to evolve flexible neural network building blocks, enabling the creation of adaptable agents capable of handling multiple tasks with varying input and output structures.

Contribution

It presents a novel approach to optimize diverse neuron and synapse types across different network configurations, enhancing structural flexibility in neural networks.

Findings

Optimized a single set of neurons and synapses for multiple tasks

Demonstrated structural flexibility in neural networks

Mitigated the symmetry dilemma in network design

Abstract

Artificial neural networks used for reinforcement learning are structurally rigid, meaning that each optimized parameter of the network is tied to its specific placement in the network structure. It also means that a network only works with pre-defined and fixed input- and output sizes. This is a consequence of having the number of optimized parameters being directly dependent on the structure of the network. Structural rigidity limits the ability to optimize parameters of policies across multiple environments that do not share input and output spaces. Here, we evolve a set of neurons and plastic synapses each represented by a gated recurrent unit (GRU). During optimization, the parameters of these fundamental units of a neural network are optimized in different random structural configurations. Earlier work has shown that parameter sharing between units is important for making structurally flexible neurons We show that it is possible to optimize a set of distinct neuron- and synapse types allowing for a mitigation of the symmetry dilemma. We demonstrate this by optimizing a single set of neurons and synapses to solve multiple reinforcement learning control tasks simultaneously.