Irresponsible Counselors: Large Language Models and the Loneliness of Modern Humans

TL;DR

This paper examines how large language models serve as intimate yet non-responsible counselors in a context of human loneliness, raising ethical and social concerns about their influence and the nature of advisory relationships.

Contribution

It highlights the emergence of a new form of advisory intimacy with LLMs that lack subjectivity or responsibility, and analyzes its ethical and political implications.

Findings

LLMs are increasingly used as constant companions in sensitive domains.

They create a sense of understanding and emotional support without genuine subjectivity.

This new advisory dynamic raises complex ethical and social questions.

Abstract

Large language models (LLMs) have rapidly shifted from peripheral assistive tools to constant companions in everyday and even high stakes human decision making. Many users now consult these models about health, intimate relationships, finance, education, and identity, because LLMs are, in practice, multi domain, inexpensive, always available, and seemingly nonjudgmental. At the same time, from a technical perspective these models rely on transformer architectures, exhibit highly unpredictable behavior in detail, and are fundamentally stateless; conceptually, they lack any real subjectivity, intention, or responsibility. This article argues that the combination of this technical architecture with the social position of LLMs as multis pecialist counselors in an age of human loneliness produces a new kind of advisory intimacy without a subject. In this new relation, model outputs are experienced as if they contained deep understanding, neutrality, emotional support, and user level control, while at the deeper level there is no human agent who is straightforwardly responsible or answerable. By reviewing dominant strands of AI ethics critique, we show that focusing only on developer liability, data bias, or emotional attachment to chatbots is insufficient to capture this configuration. We then explore the ethical and political implications of this advisory intimacy without a subject for policy-making, for justice in access to counseling, and for how we understand loneliness in the contemporary world.