Towards an Analysis of Discourse and Interactional Pragmatic Reasoning Capabilities of Large Language Models

TL;DR

This paper surveys the pragmatic reasoning capabilities of large language models, focusing on discourse and interactional pragmatics, and reviews testing methods and phenomena in these subfields.

Contribution

It provides a structured overview of pragmatic abilities tested in LLMs, categorizing phenomena and analysis methods in discourse and interactional pragmatics.

Findings

LLMs have been tested on various pragmatic phenomena

Methods for analyzing pragmatics in LLMs are diverse and heterogeneous

The survey highlights gaps and future directions in pragmatic testing

Abstract

In this work, we want to give an overview on which pragmatic abilities have been tested in LLMs so far and how these tests have been carried out. To do this, we first discuss the scope of the field of pragmatics and suggest a subdivision into discourse pragmatics and interactional pragmatics. We give a non-exhaustive overview of the phenomena of those two subdomains and the methods traditionally used to analyze them. We subsequently consider the resulting heterogeneous set of phenomena and methods as a starting point for our survey of work on discourse pragmatics and interactional pragmatics in the context of LLMs.