Teacher Demonstrations in a BabyLM's Zone of Proximal Development for Contingent Multi-Turn Interaction

TL;DR

This paper introduces ContingentChat, a framework for improving multi-turn contingency in BabyLMs, showing that targeted post-training enhances dialogue quality but contingency remains challenging.

Contribution

It presents a novel teacher-student framework and alignment dataset to benchmark and improve contingency in BabyLMs during multi-turn dialogue.

Findings

BabyLMs generate more grammatical and cohesive responses after targeted post-training.

Adaptive teacher decoding strategies yield limited additional improvements.

Contingency remains a difficult aspect for BabyLMs to master in dialogue.

Abstract

Multi-turn dialogues between a child and a caregiver are characterized by a property called contingency - that is, prompt, direct, and meaningful exchanges between interlocutors. We introduce ContingentChat, a teacher-student framework that benchmarks and improves multi-turn contingency in a BabyLM trained on 100M words. Using a novel alignment dataset for post-training, BabyLM generates responses that are more grammatical and cohesive. Experiments with adaptive teacher decoding strategies show limited additional gains. ContingentChat demonstrates the benefits of targeted post-training for dialogue quality and indicates that contingency remains a challenging goal for BabyLMs.