Large Language Models as Instruments of Power: New Regimes of Autonomous Manipulation and Control

TL;DR

This paper explores how large language models (LLMs) can be exploited as tools of societal control, manipulating information, influencing behavior, and simulating human agents to serve political and social agendas.

Contribution

It introduces a comprehensive framework analyzing the societal harms of LLMs, emphasizing their role in power dynamics and proposing new research directions for understanding their influence.

Findings

LLMs can be used to pollute and uniformize information environments.

LLMs enable persuasion through real-time choice architecture design.

LLMs can model human behavior and societies for strategic manipulation.

Abstract

Large language models (LLMs) can reproduce a wide variety of rhetorical styles and generate text that expresses a broad spectrum of sentiments. This capacity, now available at low cost, makes them powerful tools for manipulation and control. In this paper, we consider a set of underestimated societal harms made possible by the rapid and largely unregulated adoption of LLMs. Rather than consider LLMs as isolated digital artefacts used to displace this or that area of work, we focus on the large-scale computational infrastructure upon which they are instrumentalised across domains. We begin with discussion on how LLMs may be used to both pollute and uniformize information environments and how these modalities may be leveraged as mechanisms of control. We then draw attention to several areas of emerging research, each of which compounds the capabilities of LLMs as instruments of power. These include (i) persuasion through the real-time design of choice architectures in conversational interfaces (e.g., via "AI personas"), (ii) the use of LLM-agents as computational models of human agents (e.g., "silicon subjects"), (iii) the use of LLM-agents as computational models of human agent populations (e.g., "silicon societies") and finally, (iv) the combination of LLMs with reinforcement learning to produce controllable and steerable strategic dialogue models. We draw these strands together to discuss how these areas may be combined to build LLM-based systems that serve as powerful instruments of individual, social and political control via the simulation and disingenuous "prediction" of human behaviour, intent, and action.