Context Unlocks Emotions: Text-based Emotion Classification Dataset Auditing with Large Language Models

TL;DR

This paper introduces a method using large language models to add contextual information to text data, improving emotion label alignment and classification accuracy without re-annotating datasets.

Contribution

It proposes a formal definition of textual context and a prompting strategy to enhance context, demonstrating improved label alignment through human and empirical evaluations.

Findings

Enhanced context improves label alignment.

Method increases classification performance.

Human evaluation confirms better annotation consistency.

Abstract

The lack of contextual information in text data can make the annotation process of text-based emotion classification datasets challenging. As a result, such datasets often contain labels that fail to consider all the relevant emotions in the vocabulary. This misalignment between text inputs and labels can degrade the performance of machine learning models trained on top of them. As re-annotating entire datasets is a costly and time-consuming task that cannot be done at scale, we propose to use the expressive capabilities of large language models to synthesize additional context for input text to increase its alignment with the annotated emotional labels. In this work, we propose a formal definition of textual context to motivate a prompting strategy to enhance such contextual information. We provide both human and empirical evaluation to demonstrate the efficacy of the enhanced context. Our method improves alignment between inputs and their human-annotated labels from both an empirical and human-evaluated standpoint.