CSP: A Simulator For Multi-Agent Ranking Competitions

TL;DR

This paper introduces a configurable simulator that uses Large Language Models as document authors to study multi-agent ranking competitions, addressing scalability issues and reflecting the impact of AI-generated content.

Contribution

It presents a novel simulator leveraging LLMs for scalable, realistic ranking competition experiments, with analytical tools and publicly available datasets.

Findings

Generated datasets demonstrate diverse competition scenarios

Analytical tools reveal insights into ranking dynamics

Code and datasets available for further research

Abstract

In ranking competitions, document authors compete for the highest rankings by modifying their content in response to past rankings. Previous studies focused on human participants, primarily students, in controlled settings. The rise of generative AI, particularly Large Language Models (LLMs), introduces a new paradigm: using LLMs as document authors. This approach addresses scalability constraints in human-based competitions and reflects the growing role of LLM-generated content on the web-a prime example of ranking competition. We introduce a highly configurable ranking competition simulator that leverages LLMs as document authors. It includes analytical tools to examine the resulting datasets. We demonstrate its capabilities by generating multiple datasets and conducting an extensive analysis. Our code and datasets are publicly available for research.