Analogical Structure, Minimal Contextual Cues and Contrastive Distractors: Input Design for Sample-Efficient Linguistic Rule Induction

TL;DR

This paper demonstrates that structured input design inspired by cognitive principles significantly enhances sample-efficient linguistic rule learning in lightweight models, outperforming large language models in controlled tasks.

Contribution

It introduces a principled input design framework based on analogical structure, contrastive learning, and minimal cues that improves linguistic rule induction in small models.

Findings

Analogical organization is the main driver of sample efficiency.

Contrastive distractors and minimal context further improve learning.

Lightweight models outperform LLMs in few-shot settings on controlled tasks.

Abstract

Large language models achieve strong performance on many tasks, but their training makes it hard to see which properties of the input support efficient linguistic rule learning. We ask how three cognitively-inspired principles of input design support sample-efficient linguistic rule induction: analogical structure, contrastive learning, and minimal contextual cue. We also ask how their effects compare to those of LLMs on the same controlled tasks. We implement these principles in structured sentence completion tasks that test English verb alternations. Lightweight models trained on hundreds to one-thousand such examples learn the alternation rules with high F1 on these tasks. Ablation studies show that analogical organisation is the main driver of sample efficiency, and contrastive distractors and minimal context help further gains. We also evaluate zero- and few-shot LLMs on the same tasks. In this controlled setting, the lightweight models reach higher F1 with far fewer task-specific data. We treat this contrast as a comparison between learning regimes rather than a general verdict on LLMs. Our results show that careful input organisation supports sample-efficient learning of linguistic rules and reveals distinct learning signatures for trained lightweight models and prompted LLMs.