Enhancing Affinity Propagation for Improved Public Sentiment Insights

TL;DR

This paper presents an improved unsupervised sentiment analysis method using enhanced Affinity Propagation clustering combined with hierarchical clustering, outperforming traditional K-means in capturing public sentiment patterns efficiently.

Contribution

It introduces a hybrid clustering approach that refines sentiment groupings without labeled data, advancing NLP techniques for scalable public sentiment analysis.

Findings

Affinity Propagation with hierarchical clustering outperforms K-means.

The hybrid method achieves higher clustering quality scores.

Unsupervised approach reduces reliance on labeled datasets.

Abstract

With the large amount of data generated every day, public sentiment is a key factor for various fields, including marketing, politics, and social research. Understanding the public sentiment about different topics can provide valuable insights. However, most traditional approaches for sentiment analysis often depend on supervised learning, which requires a significant amount of labeled data. This makes it both expensive and time-consuming to implement. This project introduces an approach using unsupervised learning techniques, particularly Affinity Propagation (AP) clustering, to analyze sentiment. AP clustering groups text data based on natural patterns, without needing predefined cluster numbers. The paper compares AP with K-means clustering, using TF-IDF Vectorization for text representation and Principal Component Analysis (PCA) for dimensionality reduction. To enhance performance, AP is combined with Agglomerative Hierarchical Clustering. This hybrid method refines clusters further, capturing both global and local sentiment structures more effectively. The effectiveness of these methods is evaluated using the Silhouette Score, Calinski-Harabasz Score, and Davies-Bouldin Index. Results show that AP with Agglomerative Hierarchical Clustering significantly outperforms K-means. This research contributes to Natural Language Processing (NLP) by proposing a scalable and efficient unsupervised learning framework for sentiment analysis, highlighting the significant societal impact of advanced AI techniques in analyzing public sentiment without the need for extensive labeled data.