TopicImpact: Improving Customer Feedback Analysis with Opinion Units for Topic Modeling and Star-Rating Prediction

TL;DR

This paper introduces TopicImpact, a system that enhances customer feedback analysis by extracting opinion units using large language models, leading to better topic coherence, sentiment understanding, and star-rating prediction, thereby providing actionable business insights.

Contribution

The paper presents a novel approach that restructures topic modeling to operate on opinion units, improving interpretability and sentiment analysis in customer reviews.

Findings

Opinion units improve topic coherence and sentiment accuracy.

Enhanced correlation between topics, sentiments, and star ratings.

System outperforms existing models in star-rating prediction accuracy.

Abstract

We improve the extraction of insights from customer reviews by restructuring the topic modelling pipeline to operate on opinion units - distinct statements that include relevant text excerpts and associated sentiment scores. Prior work has demonstrated that such units can be reliably extracted using large language models. The result is a heightened performance of the subsequent topic modeling, leading to coherent and interpretable topics while also capturing the sentiment associated with each topic. By correlating the topics and sentiments with business metrics, such as star ratings, we can gain insights on how specific customer concerns impact business outcomes. We present our system's implementation, use cases, and advantages over other topic modeling and classification solutions. We also evaluate its effectiveness in creating coherent topics and assess methods for integrating topic and sentiment modalities for accurate star-rating prediction.