Post-training for Efficient Communication via Convention Formation

TL;DR

This paper introduces a post-training method that enhances large language models' ability to form ad-hoc communication conventions, improving their multi-turn interaction efficiency.

Contribution

The authors develop a targeted fine-tuning process using heuristically identified demonstrations to enable LLMs to form communication conventions.

Findings

Post-trained LLMs show significantly improved convention formation.

New benchmarks effectively evaluate convention formation abilities.

Models demonstrate better multi-turn interaction efficiency.

Abstract

Humans communicate with increasing efficiency in multi-turn interactions, by adapting their language and forming ad-hoc conventions. In contrast, prior work shows that LLMs do not naturally show this behavior. We develop a post-training process to develop this ability through targeted fine-tuning on heuristically identified demonstrations of convention formation. We evaluate with two new benchmarks focused on this capability. First, we design a focused, cognitively-motivated interaction benchmark that consistently elicits strong convention formation trends in humans. Second, we create a new document-grounded reference completion task that reflects in-the-wild convention formation behavior. Our studies show significantly improved convention formation abilities in post-trained LLMs across the two evaluation methods.