"Is the Pope Catholic?" Applying Chain-of-Thought Reasoning to Understanding Conversational Implicatures

TL;DR

This paper shows that integrating Grice's Four Maxims into chain-of-thought prompting significantly improves large language models' ability to understand conversational implicatures, surpassing human performance.

Contribution

It introduces a novel method of applying Grice's Maxims via chain-of-thought prompting to enhance LLMs' pragmatic reasoning skills.

Findings

Enhanced model performance on implicature tasks

Models surpass average human understanding

Chain-of-thought prompting effectively encodes pragmatic principles

Abstract

Conversational implicatures are pragmatic inferences that require listeners to deduce the intended meaning conveyed by a speaker from their explicit utterances. Although such inferential reasoning is fundamental to human communication, recent research indicates that large language models struggle to comprehend these implicatures as effectively as the average human. This paper demonstrates that by incorporating Grice's Four Maxims into the model through chain-of-thought prompting, we can significantly enhance its performance, surpassing even the average human performance on this task.