Combat AI With AI: Counteract Machine-Generated Fake Restaurant Reviews on Social Media

TL;DR

This paper presents a method to detect machine-generated fake restaurant reviews by fine-tuning a GPT-based detector, leveraging verified elite reviews to improve accuracy and analyze review patterns.

Contribution

It introduces a fine-tuned GPT detector trained on verified reviews to effectively identify AI-generated fake reviews, outperforming existing detection methods.

Findings

The detector significantly outperforms previous solutions.

Patterns in fake reviews differ across review, user, and restaurant features.

Detection models can identify AI-generated reviews with high accuracy.

Abstract

Recent advances in generative models such as GPT may be used to fabricate indistinguishable fake customer reviews at a much lower cost, thus posing challenges for social media platforms to detect these machine-generated fake reviews. We propose to leverage the high-quality elite restaurant reviews verified by Yelp to generate fake reviews from the OpenAI GPT review creator and ultimately fine-tune a GPT output detector to predict fake reviews that significantly outperform existing solutions. We further apply the model to predict non-elite reviews and identify the patterns across several dimensions, such as review, user and restaurant characteristics, and writing style. We show that social media platforms are continuously challenged by machine-generated fake reviews, although they may implement detection systems to filter out suspicious reviews.