AnthroScore: A Computational Linguistic Measure of Anthropomorphism

TL;DR

AnthroScore is an automatic, lexicon-free metric that quantifies implicit anthropomorphism in language, revealing increasing trends in research and media, and correlating with human judgments and social science dimensions.

Contribution

We introduce AnthroScore, a novel computational metric for implicit anthropomorphism that aligns with social science theories and can analyze large text corpora over time.

Findings

AnthroScore correlates with human judgments of anthropomorphism.

Anthropomorphism in research papers has increased over 15 years.

News headlines show higher anthropomorphism than cited research articles.

Abstract

Anthropomorphism, or the attribution of human-like characteristics to non-human entities, has shaped conversations about the impacts and possibilities of technology. We present AnthroScore, an automatic metric of implicit anthropomorphism in language. We use a masked language model to quantify how non-human entities are implicitly framed as human by the surrounding context. We show that AnthroScore corresponds with human judgments of anthropomorphism and dimensions of anthropomorphism described in social science literature. Motivated by concerns of misleading anthropomorphism in computer science discourse, we use AnthroScore to analyze 15 years of research papers and downstream news articles. In research papers, we find that anthropomorphism has steadily increased over time, and that papers related to language models have the most anthropomorphism. Within ACL papers, temporal increases in anthropomorphism are correlated with key neural advancements. Building upon concerns of scientific misinformation in mass media, we identify higher levels of anthropomorphism in news headlines compared to the research papers they cite. Since AnthroScore is lexicon-free, it can be directly applied to a wide range of text sources.