Animate, or Inanimate, That is the Question for Large Language Models

TL;DR

This paper investigates whether large language models can understand animacy, a concept crucial in human cognition, by systematically probing their responses to animate and inanimate entities through prompting, revealing human-like behavior in LLMs.

Contribution

The study provides a systematic analysis of LLMs' ability to process animacy, showing they can recognize animate and inanimate entities similarly to humans despite training only on text.

Findings

LLMs exhibit human-like recognition of animate and inanimate entities.

LLMs can adapt to unconventional animacy contexts.

Prompting effectively reveals LLMs' understanding of animacy.

Abstract

The cognitive essence of humans is deeply intertwined with the concept of animacy, which plays an essential role in shaping their memory, vision, and multi-layered language understanding. Although animacy appears in language via nuanced constraints on verbs and adjectives, it is also learned and refined through extralinguistic information. Similarly, we assume that the LLMs' limited abilities to understand natural language when processing animacy are motivated by the fact that these models are trained exclusively on text. Hence, the question this paper aims to answer arises: can LLMs, in their digital wisdom, process animacy in a similar way to what humans would do? We then propose a systematic analysis via prompting approaches. In particular, we probe different LLMs by prompting them using animate, inanimate, usual, and stranger contexts. Results reveal that, although LLMs have been trained predominantly on textual data, they exhibit human-like behavior when faced with typical animate and inanimate entities in alignment with earlier studies. Hence, LLMs can adapt to understand unconventional situations by recognizing oddities as animated without needing to interface with unspoken cognitive triggers humans rely on to break down animations.