Why is "Chicago" Predictive of Deceptive Reviews? Using LLMs to Discover Language Phenomena from Lexical Cues

TL;DR

This paper explores how large language models can translate subtle lexical cues in deceptive reviews into understandable language phenomena, aiding human judgment and trust in online reviews.

Contribution

It introduces a conjecture-then-validate framework to identify empirically grounded language phenomena that improve deception detection interpretability.

Findings

Language phenomena are empirically grounded and generalizable.

Phenomena derived are more predictive than prior knowledge-based ones.

The framework helps humans critically assess review credibility.

Abstract

Deceptive reviews mislead consumers, harm businesses, and undermine trust in online marketplaces. Machine learning classifiers can learn from large amounts of data to distinguish deceptive reviews from genuine ones. However, the distinguishing features learned by these classifiers are often subtle, fragmented, and difficult for humans to interpret, which can hinder user understanding and trust. In this work, we study whether large language models (LLMs) can translate such unintuitive lexical cues into human-understandable language phenomena. We propose a conjecture-then-validate framework, and show that language phenomena obtained in this manner are empirically grounded in data, generalizable across similar domains, and more predictive than phenomena derived from LLMs' prior knowledge or in-context learning. Such phenomena can aid people in critically assessing the credibility of online reviews in environments where deception detection classifiers are unavailable.