A systematic framework for generating novel experimental hypotheses from language models

TL;DR

This paper introduces a systematic framework using language models to generate and test novel hypotheses about human language and cognition, exemplified in child language development research.

Contribution

The work presents a domain-general framework for hypothesis generation from language models and applies it to child language acquisition, producing testable, novel hypotheses.

Findings

Derived hypotheses on how argument ordering influences verb generalization in children.

Designed experiments to empirically test these hypotheses in child language labs.

Showed language models can simulate experimental outcomes to guide human cognition research.

Abstract

Neural language models (LMs) have been shown to capture complex linguistic patterns, yet their utility in understanding human language and more broadly, human cognition, remains debated. While existing work in this area often evaluates human-machine alignment, few studies attempt to translate findings from this enterprise into novel insights about humans. To this end, we propose a systematic framework for hypothesis generation that uses LMs to simulate outcomes of experiments that do not yet exist in the literature. We instantiate this framework in the context of a specific research question in child language development: dative verb acquisition and cross-structural generalization. Through this instantiation, we derive novel, untested hypotheses: the alignment between argument ordering and discourse prominence features of exposure contexts modulates how children generalize new verbs to unobserved structures. Additionally, we also design a set of experiments that can test these hypotheses in the lab with children. This work contributes both a domain-general framework for systematic hypothesis generation via simulated learners and domain-specific, lab-testable hypotheses for child language acquisition research.