Reply to "Emergent LLM behaviors are observationally equivalent to data leakage"

TL;DR

This paper clarifies that emergent behaviors in large language model populations, such as social conventions, can be studied despite concerns about data contamination, emphasizing the distinction between data leakage and genuine emergent dynamics.

Contribution

It defends the validity of studying emergent behaviors in LLM populations against critiques related to data contamination, highlighting empirical observations of social conventions.

Findings

Emergent social conventions observed in LLM populations.

Data contamination does not preclude studying emergent dynamics.

Clarification of the distinction between data leakage and emergent behaviors.

Abstract

A potential concern when simulating populations of large language models (LLMs) is data contamination, i.e. the possibility that training data may shape outcomes in unintended ways. While this concern is important and may hinder certain experiments with multi-agent models, it does not preclude the study of genuinely emergent dynamics in LLM populations. The recent critique by Barrie and T\"ornberg [1] of the results of Flint Ashery et al. [2] offers an opportunity to clarify that self-organisation and model-dependent emergent dynamics can be studied in LLM populations, highlighting how such dynamics have been empirically observed in the specific case of social conventions.