Measuring Fine-Grained Negotiation Tactics of Humans and LLMs in Diplomacy

TL;DR

This paper analyzes negotiation styles in the game Diplomacy, using large language models as judges to annotate tactics, comparing human and AI strategies, and exploring how fine-tuning can make AI more human-like in negotiations.

Contribution

It introduces a sociologically-grounded taxonomy for negotiation tactics, uses LLMs as judges for large datasets, and compares human and AI negotiation strategies in Diplomacy.

Findings

LLMs reliably annotate negotiation tactics.

Strong correlation between tactics and game success.

Fine-tuning improves AI's human-like negotiation behavior.

Abstract

The study of negotiation styles dates back to Aristotle's ethos-pathos-logos rhetoric. Prior efforts primarily studied the success of negotiation agents. Here, we shift the focus towards the styles of negotiation strategies. Our focus is the strategic dialogue board game Diplomacy, which affords rich natural language negotiation and measures of game success. We used LLM-as-a-judge to annotate a large human-human set of Diplomacy games for fine-grained negotiation tactics from a sociologically-grounded taxonomy. Using a combination of the It Takes Two and WebDiplomacy datasets, we demonstrate the reliability of our LLM-as-a-Judge framework and show strong correlations between negotiation features and success in the Diplomacy setting. Lastly, we investigate the differences between LLM and human negotiation strategies and show that fine-tuning can steer LLM agents toward more human-like negotiation behaviors.