Learning to Make Analogies by Contrasting Abstract Relational Structure

TL;DR

This paper demonstrates that neural networks can learn to perform complex analogical reasoning on visual and symbolic data by contrasting abstract relational structures, emphasizing data presentation over architecture complexity.

Contribution

It introduces a training method that induces analogical reasoning in neural networks through contrasting abstract relational structures, without requiring elaborate architectures.

Findings

Neural networks can perform complex analogy making and generalization.

Contrasting abstract relational structures is key to inducing analogical reasoning.

Simple neural architectures can learn to reason analogically with proper training.

Abstract

Analogical reasoning has been a principal focus of various waves of AI research. Analogy is particularly challenging for machines because it requires relational structures to be represented such that they can be flexibly applied across diverse domains of experience. Here, we study how analogical reasoning can be induced in neural networks that learn to perceive and reason about raw visual data. We find that the critical factor for inducing such a capacity is not an elaborate architecture, but rather, careful attention to the choice of data and the manner in which it is presented to the model. The most robust capacity for analogical reasoning is induced when networks learn analogies by contrasting abstract relational structures in their input domains, a training method that uses only the input data to force models to learn about important abstract features. Using this technique we demonstrate capacities for complex, visual and symbolic analogy making and generalisation in even the simplest neural network architectures.