The structure of evolved representations across different substrates for artificial intelligence

TL;DR

This paper compares different neural network substrates, revealing that Markov Brains localize information making them more robust to noise, unlike recurrent neural networks and LSTMs which distribute information broadly and are more vulnerable.

Contribution

It provides a comparative analysis of information representation in various neural substrates, highlighting the robustness of Markov Brains due to localized information storage.

Findings

Markov Brains localize and sparsely distribute information

Recurrent neural networks and LSTMs spread information across nodes

Markov Brains are more resistant to noise

Abstract

Artificial neural networks (ANNs), while exceptionally useful for classification, are vulnerable to misdirection. Small amounts of noise can significantly affect their ability to correctly complete a task. Instead of generalizing concepts, ANNs seem to focus on surface statistical regularities in a given task. Here we compare how recurrent artificial neural networks, long short-term memory units, and Markov Brains sense and remember their environments. We show that information in Markov Brains is localized and sparsely distributed, while the other neural network substrates "smear" information about the environment across all nodes, which makes them vulnerable to noise.