Unveiling Dual Quality in Product Reviews: An NLP-Based Approach

TL;DR

This paper develops NLP-based methods to detect dual quality issues in product reviews, creating a Polish dataset and evaluating multiple models, including multilingual transfer, to improve automated identification of inconsistent product quality.

Contribution

Introduces a new Polish-language dataset for dual quality detection and evaluates various NLP models, including multilingual transfer, for identifying quality discrepancies in reviews.

Findings

SetFit with sentence-transformers performs well in detection tasks.

Multilingual transfer shows promising results across languages.

Error analysis highlights key challenges in robustness and accuracy.

Abstract

Consumers often face inconsistent product quality, particularly when identical products vary between markets, a situation known as the dual quality problem. To identify and address this issue, automated techniques are needed. This paper explores how natural language processing (NLP) can aid in detecting such discrepancies and presents the full process of developing a solution. First, we describe in detail the creation of a new Polish-language dataset with 1,957 reviews, 540 highlighting dual quality issues. We then discuss experiments with various approaches like SetFit with sentence-transformers, transformer-based encoders, and LLMs, including error analysis and robustness verification. Additionally, we evaluate multilingual transfer using a subset of opinions in English, French, and German. The paper concludes with insights on deployment and practical applications.