Can structural correspondences ground real world representational content in Large Language Models?

TL;DR

This paper investigates whether structural correspondences in Large Language Models can ground real-world content, arguing that mere structural similarity is insufficient unless it is exploited for task performance, addressing the challenge of text-boundedness.

Contribution

It offers a structural-correspondence account of representation in LLMs and surveys evidence for their capacity to ground real-world content.

Findings

Structural correspondences alone do not ground representation.

Exploiting structural correspondences can ground real-world content.

Overcoming text-boundedness is crucial for grounding in LLMs.

Abstract

Large Language Models (LLMs) such as GPT-4 produce compelling responses to a wide range of prompts. But their representational capacities are uncertain. Many LLMs have no direct contact with extra-linguistic reality: their inputs, outputs and training data consist solely of text, raising the questions (1) can LLMs represent anything and (2) if so, what? In this paper, I explore what it would take to answer these questions according to a structural-correspondence based account of representation, and make an initial survey of this evidence. I argue that the mere existence of structural correspondences between LLMs and worldly entities is insufficient to ground representation of those entities. However, if these structural correspondences play an appropriate role - they are exploited in a way that explains successful task performance - then they could ground real world contents. This requires overcoming a challenge: the text-boundedness of LLMs appears, on the face of it, to prevent them engaging in the right sorts of tasks.