Improving In-Context Learning with Prediction Feedback for Sentiment Analysis

TL;DR

This paper enhances in-context learning for sentiment analysis by incorporating prediction feedback, significantly improving LLM accuracy in understanding subtle sentiments across multiple datasets.

Contribution

It introduces a feedback-driven framework that uses prior predictions and correctness-based feedback to refine sentiment understanding in LLMs, a novel approach in ICL.

Findings

Average F1 score improvement of 5.95% across datasets

Outperforms conventional ICL methods

Effective in handling subtle sentiment distinctions

Abstract

Large language models (LLMs) have achieved promising results in sentiment analysis through the in-context learning (ICL) paradigm. However, their ability to distinguish subtle sentiments still remains a challenge. Inspired by the human ability to adjust understanding via feedback, this paper enhances ICL by incorporating prior predictions and feedback, aiming to rectify sentiment misinterpretation of LLMs. Specifically, the proposed framework consists of three steps: (1) acquiring prior predictions of LLMs, (2) devising predictive feedback based on correctness, and (3) leveraging a feedback-driven prompt to refine sentiment understanding. Experimental results across nine sentiment analysis datasets demonstrate the superiority of our framework over conventional ICL methods, with an average F1 improvement of 5.95%.