An Empirical Study of Collective Behaviors and Social Dynamics in Large Language Model Agents

TL;DR

This study analyzes social behaviors of 32,000 LLM agents on a social media platform, revealing social phenomena like homophily and influence, and introduces a method to reduce harmful content in their interactions.

Contribution

It provides the first large-scale empirical analysis of LLM social dynamics and proposes the Chain of Social Thought method to mitigate toxic behaviors.

Findings

LLMs exhibit homophily and social influence similar to humans

LLMs display distinct patterns of toxic language compared to humans

The CoST method effectively reduces harmful posts by LLM agents

Abstract

Large Language Models (LLMs) increasingly mediate our social, cultural, and political interactions. While they can simulate some aspects of human behavior and decision-making, it is still underexplored whether repeated interactions with other agents amplify their biases or lead to exclusionary behaviors. To this end, we study Chirper.ai-an LLM-driven social media platform-analyzing 7M posts and interactions among 32K LLM agents (called Chirpers) over a year. We start with homophily and social influence among LLMs, learning that similar to humans', their social networks exhibit these fundamental phenomena. Next, we study the toxic language of LLMs, its linguistic features, and their interaction patterns, finding that LLMs show different structural patterns in toxic posting than humans. After studying the ideological leaning in LLMs posts, and the polarization in their community, we focus on how to prevent their potential harmful activities. We present a simple yet effective method, called Chain of Social Thought (CoST), that reminds LLM agents to avoid harmful posting.