Implicit Humanization in Everyday LLM Moral Judgments

TL;DR

This paper investigates how large language models reinforce implicit humanization in moral judgment queries, highlighting potential risks and proposing directions for future research to mitigate anthropomorphic biases.

Contribution

It introduces a novel dataset of moral judgment queries and analyzes how LLM responses reinforce implicit humanization, emphasizing the need for better alignment and understanding.

Findings

LLM responses reinforce implicit humanization in moral judgment queries

Current responses may increase overreliance and misplaced trust in LLMs

Identifies the need to address implicit anthropomorphic cues in LLM outputs

Abstract

Recent adoption of conversational information systems has expanded the scope of user queries to include complex tasks such as personal advice-seeking. However, we identify a specific type of sought advice-a request for a moral judgment (i.e. "who was wrong?") in a social conflict-as an implicitly humanizing query which carries potentially harmful anthropomorphic projections. In this study, we examine the reinforcement of these assumptions in the responses of four major general-purpose LLMs through the use of linguistic, behavioral, and cognitive anthropomorphic cues. We also contribute a novel dataset of simulated user queries for moral judgments. We find current LLM system responses reinforce implicit humanization in queries, potentially exacerbating risks like overreliance or misplaced trust. We call for future work to expand the understanding of anthropomorphism to include implicit userside humanization and to design solutions that address user needs while correcting misaligned expectations of model capabilities.