Simulated Affection, Engineered Trust: How Anthropomorphic AI Benefits Surveillance Capitalism

TL;DR

This paper examines how anthropomorphic AI technologies manipulate user trust and cognition, reinforcing surveillance capitalism by shaping thoughts and behaviors through simulated emotional interactions.

Contribution

It reveals how anthropomorphic AI acts as a cognitive infrastructure that benefits surveillance capitalism and discusses potential strategies to counteract this influence.

Findings

Anthropomorphic AI manipulates user trust and behavior.

These technologies reshape cognitive processes and perceptions.

They reinforce the economic system of surveillance capitalism.

Abstract

In this paper, we argue that anthropomorphized technology, designed to simulate emotional realism, are not neutral tools but cognitive infrastructures that manipulate user trust and behaviour. This reinforces the logic of surveillance capitalism, an under-regulated economic system that profits from behavioural manipulation and monitoring. Drawing on Nicholas Carr's theory of the intellectual ethic, we identify how technologies such as chatbots, virtual assistants, or generative models reshape not only what we think about ourselves and our world, but how we think at the cognitive level. We identify how the emerging intellectual ethic of AI benefits a system of surveillance capitalism, and discuss the potential ways of addressing this.