Reference Points in LLM Sentiment Analysis: The Role of Structured Context

TL;DR

This study shows that structured prompts, like JSON format, improve sentiment analysis accuracy in LLMs by incorporating reference points, making smaller models more effective without fine-tuning.

Contribution

It demonstrates that structured prompting with JSON enhances LLM sentiment analysis performance, enabling resource-efficient deployment without model fine-tuning.

Findings

JSON prompts outperform natural language prompts in sentiment classification.

Performance gains are due to improved contextual reasoning, not label proxying.

Small models with structured prompts can rival larger models in accuracy.

Abstract

Large language models (LLMs) are now widely used across many fields, including marketing research. Sentiment analysis, in particular, helps firms understand consumer preferences. While most NLP studies classify sentiment from review text alone, marketing theories, such as prospect theory and expectation--disconfirmation theory, point out that customer evaluations are shaped not only by the actual experience but also by additional reference points. This study therefore investigates how the content and format of such supplementary information affect sentiment analysis using LLMs. We compare natural language (NL) and JSON-formatted prompts using a lightweight 3B parameter model suitable for practical marketing applications. Experiments on two Yelp categories (Restaurant and Nightlife) show that the JSON prompt with additional information outperforms all baselines without fine-tuning: Macro-F1 rises by 1.6% and 4% while RMSE falls by 16% and 9.1%, respectively, making it deployable in resource-constrained edge devices. Furthermore, a follow-up analysis confirms that performance gains stem from genuine contextual reasoning rather than label proxying. This work demonstrates that structured prompting can enable smaller models to achieve competitive performance, offering a practical alternative to large-scale model deployment.