Comparing Without Saying: A Dataset and Benchmark for Implicit Comparative Opinion Mining from Same-User Reviews

TL;DR

This paper introduces SUDO, a new dataset for implicit comparative opinion mining from same-user reviews, enabling the study of user preferences without explicit comparison cues, and benchmarks current models on this challenging task.

Contribution

The paper presents SUDO, a novel dataset with annotated review pairs for implicit comparison analysis, and provides baseline evaluations highlighting the task's difficulty.

Findings

Language models outperform traditional methods

Overall performance remains moderate

SUDO is a challenging benchmark for future research

Abstract

Existing studies on comparative opinion mining have mainly focused on explicit comparative expressions, which are uncommon in real-world reviews. This leaves implicit comparisons - here users express preferences across separate reviews - largely underexplored. We introduce SUDO, a novel dataset for implicit comparative opinion mining from same-user reviews, allowing reliable inference of user preferences even without explicit comparative cues. SUDO comprises 4,150 annotated review pairs (15,191 sentences) with a bi-level structure capturing aspect-level mentions and review-level preferences. We benchmark this task using two baseline architectures: traditional machine learning- and language model-based baselines. Experimental results show that while the latter outperforms the former, overall performance remains moderate, revealing the inherent difficulty of the task and establishing SUDO as a challenging and valuable benchmark for future research.