Abstraction, Reasoning and Deep Learning: A Study of the "Look and Say" Sequence

TL;DR

This study investigates whether deep neural networks truly understand the 'Look and Say' sequence, finding they excel at accuracy but lack genuine comprehension of the underlying principles, highlighting limitations in current AI models.

Contribution

The paper demonstrates that neural networks trained on large data sets do not exhibit true understanding of the 'Look and Say' sequence, challenging assumptions about AI's grasp of abstract concepts.

Findings

Neural networks perform poorly on sequence tasks despite high accuracy.

Current models lack genuine understanding of sequence principles.

Implications for cognitive science and neural network theory.

Abstract

The ability to abstract, count, and use System~2 reasoning are well-known manifestations of intelligence and understanding. In this paper, we argue, using the example of the ``Look and Say" puzzle, that although deep neural networks can exhibit high `competence' (as measured by accuracy) when trained on large data sets (2 million examples in our case), they do not show any sign on the deeper understanding of the problem, or what D. Dennett calls `comprehension'. We report on two sets experiments: first, computing the next element of the sequence, and ,then, the previous element. We view both problems as building a translator from one set of tokens to another. We apply both standard LSTMs and Transformer/Attention-based neural networks, using publicly available machine translation software. We observe that despite the amazing accuracy, the performance of the trained programs on the actual L\&S sequence is bad, and shows no understanding of the principles behind the sequences. The ramifications of this finding include: (1) from the cognitive science perspective, we argue that we need better mathematical models of abstraction; (2) the universality of neural networks should be re-examined for functions acting on discrete data sets; (3) we hypothesize topology can provide a definition of without the reference to the concept of distance.