A Neuroscience-Inspired Dual-Process Model of Compositional Generalization

TL;DR

This paper introduces Mirage, a neuro-inspired dual-process model combining fast intuitive and deliberate rule-based systems, achieving near-perfect compositional generalization in language tasks, and providing insights into cognitive architecture.

Contribution

Mirage is the first neuro-inspired dual-process model that effectively combines neural and symbolic components for systematic compositional generalization.

Findings

Achieves >99% accuracy on SCAN benchmark splits

Systematic behavior emerges from the interaction of the two systems

Externalized schemas enable iterative refinement and interpretability

Abstract

Deep learning models struggle with systematic compositional generalization, a hallmark of human cognition. We propose \textsc{Mirage}, a neuro-inspired dual-process model that offers a processing account for this ability. It combines a fast, intuitive ``System~1'' (a meta-trained Transformer) with a deliberate, rule-based ``System~2'' (a Schema Engine), mirroring the brain's neocortical and hippocampal--prefrontal circuits. Trained to perform general, single-step decomposition on a stream of random grammars, Mirage achieves $>$99\% accuracy on all splits of the SCAN benchmark in a task-agnostic setting. Ablations confirm that the model's systematic behavior emerges from the architectural interplay of its two systems, particularly its use of explicit, prioritized schemas and iterative refinement. In line with recent progress on recursive/recurrent Transformer approaches, Mirage preserves an iterative neural update while externalizing declarative control into an interpretable schema module. Our work provides a concrete computational model for interpreting how compositional reasoning can arise from a modular cognitive architecture.