Leveraging Machine-Generated Rationales to Facilitate Social Meaning Detection in Conversations

TL;DR

This paper introduces a method using Large Language Models to generate rationales that improve the detection of social meanings in conversations, demonstrating significant benefits across various settings and domains.

Contribution

It presents a novel approach that leverages LLM-generated rationales to enhance social meaning detection, showing effectiveness in multiple experimental scenarios.

Findings

Rationales significantly improve classification accuracy.

Method works well in zero-shot and few-shot transfer settings.

Effective across different social meaning detection tasks and datasets.

Abstract

We present a generalizable classification approach that leverages Large Language Models (LLMs) to facilitate the detection of implicitly encoded social meaning in conversations. We design a multi-faceted prompt to extract a textual explanation of the reasoning that connects visible cues to underlying social meanings. These extracted explanations or rationales serve as augmentations to the conversational text to facilitate dialogue understanding and transfer. Our empirical results over 2,340 experimental settings demonstrate the significant positive impact of adding these rationales. Our findings hold true for in-domain classification, zero-shot, and few-shot domain transfer for two different social meaning detection tasks, each spanning two different corpora.