Improving Multilingual Social Media Insights: Aspect-based Comment Analysis

TL;DR

This paper introduces a multilingual comment aspect term generation method using fine-tuned large language models, improving social media discourse understanding across multiple languages and providing a new multilingual test set.

Contribution

It presents a novel multilingual aspect term generation approach with supervised fine-tuning and DPO, along with the first multilingual test set for English, Chinese, Malay, and Bahasa Indonesian.

Findings

Enhanced social media comment analysis performance

Effective cross-lingual comparison of LLMs

First multilingual CAT-G test set introduced

Abstract

The inherent nature of social media posts, characterized by the freedom of language use with a disjointed array of diverse opinions and topics, poses significant challenges to downstream NLP tasks such as comment clustering, comment summarization, and social media opinion analysis. To address this, we propose a granular level of identifying and generating aspect terms from individual comments to guide model attention. Specifically, we leverage multilingual large language models with supervised fine-tuning for comment aspect term generation (CAT-G), further aligning the model's predictions with human expectations through DPO. We demonstrate the effectiveness of our method in enhancing the comprehension of social media discourse on two NLP tasks. Moreover, this paper contributes the first multilingual CAT-G test set on English, Chinese, Malay, and Bahasa Indonesian. As LLM capabilities vary among languages, this test set allows for a comparative analysis of performance across languages with varying levels of LLM proficiency.