The Relational Bottleneck as an Inductive Bias for Efficient Abstraction

TL;DR

This paper discusses the relational bottleneck as an architectural inductive bias in neural networks that promotes efficient abstraction of relational concepts, bridging connectionist and symbolic approaches in cognitive modeling.

Contribution

It introduces the relational bottleneck as a novel architectural bias that enhances data-efficient abstraction of relations, offering a potential model for human concept acquisition.

Findings

Models with the relational bottleneck induce abstractions efficiently.

The approach bridges connectionist and symbolic cognitive models.

Potential relevance for understanding human abstract concept learning.

Abstract

A central challenge for cognitive science is to explain how abstract concepts are acquired from limited experience. This has often been framed in terms of a dichotomy between connectionist and symbolic cognitive models. Here, we highlight a recently emerging line of work that suggests a novel reconciliation of these approaches, by exploiting an inductive bias that we term the relational bottleneck. In that approach, neural networks are constrained via their architecture to focus on relations between perceptual inputs, rather than the attributes of individual inputs. We review a family of models that employ this approach to induce abstractions in a data-efficient manner, emphasizing their potential as candidate models for the acquisition of abstract concepts in the human mind and brain.