LLMs' Understanding of Natural Language Revealed

TL;DR

This paper critically examines the language understanding capabilities of large language models (LLMs), revealing that their supposed understanding is superficial and primarily based on memorization rather than genuine comprehension.

Contribution

The study introduces a novel testing approach that assesses LLMs' understanding by querying them with snippets of text, demonstrating their limited true language comprehension.

Findings

LLMs' understanding is superficial and based on memorization.

Traditional evaluations overestimate LLMs' language understanding.

LLMs perform poorly on understanding tasks that require genuine comprehension.

Abstract

Large language models (LLMs) are the result of a massive experiment in bottom-up, data-driven reverse engineering of language at scale. Despite their utility in a number of downstream NLP tasks, ample research has shown that LLMs are incapable of performing reasoning in tasks that require quantification over and the manipulation of symbolic variables (e.g., planning and problem solving); see for example [25][26]. In this document, however, we will focus on testing LLMs for their language understanding capabilities, their supposed forte. As we will show here, the language understanding capabilities of LLMs have been widely exaggerated. While LLMs have proven to generate human-like coherent language (since that's how they were designed), their language understanding capabilities have not been properly tested. In particular, we believe that the language understanding capabilities of LLMs should be tested by performing an operation that is the opposite of 'text generation' and specifically by giving the LLM snippets of text as input and then querying what the LLM "understood". As we show here, when doing so it will become apparent that LLMs do not truly understand language, beyond very superficial inferences that are essentially the byproduct of the memorization of massive amounts of ingested text.