Can we automatize scientific discovery in the cognitive sciences?

TL;DR

This paper proposes a fully automated, in silico approach to scientific discovery in cognitive sciences using Large Language Models to generate hypotheses, simulate data, and evaluate theories, significantly accelerating the research cycle.

Contribution

It introduces a novel framework that automates the entire discovery process in cognitive sciences with LLMs, reducing manual effort and expanding exploration capacity.

Findings

Automated generation of experimental paradigms from LLMs.

High-throughput simulation of behavioral data using foundation models.

Optimization of theories based on an LLM-critic evaluating interestingness.

Abstract

The cognitive sciences aim to understand intelligence by formalizing underlying operations as computational models. Traditionally, this follows a cycle of discovery where researchers develop paradigms, collect data, and test predefined model classes. However, this manual pipeline is fundamentally constrained by the slow pace of human intervention and a search space limited by researchers' background and intuition. Here, we propose a paradigm shift toward a fully automated, in silico science of the mind that implements every stage of the discovery cycle using Large Language Models (LLMs). In this framework, experimental paradigms exploring conceptually meaningful task structures are directly sampled from an LLM. High-fidelity behavioral data are then simulated using foundation models of cognition. The tedious step of handcrafting cognitive models is replaced by LLM-based program synthesis, which performs a high-throughput search over a vast landscape of algorithmic hypotheses. Finally, the discovery loop is closed by optimizing for ''interestingness'', a metric of conceptual yield evaluated by an LLM-critic. By enabling a fast and scalable approach to theory development, this automated loop functions as a high-throughput in-silico discovery engine, surfacing informative experiments and mechanisms for subsequent validation in real human populations.