Bootstrapping Concept Formation in Small Neural Networks

TL;DR

This paper presents a neural network model demonstrating how small systems can form concept-like relational representations through associative learning, providing insights into primitive concept formation in biological and artificial agents.

Contribution

The study introduces a neural network model that forms relational concepts via associative learning using realistic rules and minimal feedback, highlighting a potential mechanism for primitive concept formation.

Findings

Neuronal pools represent relational information.

Relational pools influence agent behavior via feedback.

Associative learning suffices for concept-like structure development.

Abstract

The question how neural systems (of humans) can perform reasoning is still far from being solved. We posit that the process of forming Concepts is a fundamental step required for this. We argue that, first, Concepts are formed as closed representations, which are then consolidated by relating them to each other. Here we present a model system (agent) with a small neural network that uses realistic learning rules and receives only feedback from the environment in which the agent performs virtual actions. First, the actions of the agent are reflexive. In the process of learning, statistical regularities in the input lead to the formation of neuronal pools representing relations between the entities observed by the agent from its artificial world. This information then influences the behavior of the agent via feedback connections replacing the initial reflex by an action driven by these relational representations. We hypothesize that the neuronal pools representing relational information can be considered as primordial Concepts, which may in a similar way be present in some pre-linguistic animals, too. This system provides formal grounds for further discussions on what could be understood as a Concept and shows that associative learning is enough to develop concept-like structures.