Fodor and Pylyshyn's Legacy: Still No Human-like Systematic Compositionality in Neural Networks

TL;DR

This paper critically examines the claim that neural networks can develop human-like systematic compositionality, concluding that current meta-learning approaches fall short of this goal and that Fodor and Pylyshyn's skepticism remains valid.

Contribution

The paper provides a critical analysis of meta-learning frameworks for compositionality, highlighting their limitations and reaffirming the absence of human-like systematic compositionality in neural networks.

Findings

Neural meta-learning systems perform compositional tasks only under narrow conditions.

Current models do not exhibit human-like systematic compositionality.

Fodor and Pylyshyn's skepticism about neural networks' compositionality remains supported.

Abstract

Strong meta-learning capabilities for systematic compositionality are emerging as an important skill for navigating the complex and changing tasks of today's world. However, in presenting models for robust adaptation to novel environments, it is important to refrain from making unsupported claims about the performance of meta-learning systems that ultimately do not stand up to scrutiny. While Fodor and Pylyshyn famously posited that neural networks inherently lack this capacity as they are unable to model compositional representations or structure-sensitive operations, and thus are not a viable model of the human mind, Lake and Baroni recently presented meta-learning as a pathway to compositionality. In this position paper, we critically revisit this claim and highlight limitations in the proposed meta-learning framework for compositionality. Our analysis shows that modern neural meta-learning systems can only perform such tasks, if at all, under a very narrow and restricted definition of a meta-learning setup. We therefore claim that `Fodor and Pylyshyn's legacy' persists, and to date, there is no human-like systematic compositionality learned in neural networks.