Whither symbols in the era of advanced neural networks?

TL;DR

This paper argues that modern neural networks demonstrate abilities similar to symbolic reasoning, challenging the view that human cognition is fundamentally symbolic, and suggests new directions for understanding the symbolic basis of thought.

Contribution

It presents a perspective that neural networks exhibit symbolic-like abilities, prompting a reevaluation of the role of symbols in human cognition research.

Findings

Neural networks can combine ideas and produce novelty.

Neural networks learn quickly, similar to symbolic systems.

Symbolic systems remain relevant in understanding neural network training data.

Abstract

Some of the strongest evidence that human minds should be thought about in terms of symbolic systems has been the way they combine ideas, produce novelty, and learn quickly. We argue that modern neural networks -- and the artificial intelligence systems built upon them -- exhibit similar abilities. This undermines the argument that the cognitive processes and representations used by human minds are symbolic, although the fact that these neural networks are typically trained on data generated by symbolic systems illustrates that such systems play an important role in characterizing the abstract problems that human minds have to solve. This argument leads us to offer a new agenda for research on the symbolic basis of human thought.