Anticipate, Simulate, Reason (ASR): A Comprehensive Generative AI Framework for Combating Messaging Scams

TL;DR

This paper introduces the ASR generative AI framework that predicts scammer responses and aids users in identifying messaging scams, significantly improving detection and understanding of scam tactics.

Contribution

It presents a novel AI framework and a domain-specific language model, ScamGPT-J, for proactive scam detection and interpretability in messaging platforms.

Findings

Enhanced scam detection accuracy, especially in job scams

Revealed demographic vulnerabilities and perceptions of AI support

Contradiction between at-risk users and receptiveness to AI aid

Abstract

The rapid growth of messaging scams creates an escalating challenge for user security and financial safety. In this paper, we present the \textit{Anticipate, Simulate, Reason} (ASR) generative AI framework to enable users to proactively identify and comprehend scams within instant messaging platforms. Using large language models, ASR predicts scammer responses and delivers real-time, interpretable support to end-users. We also develop ScamGPT-J, a domain-specific language model fine-tuned on a new, high-quality dataset of scam conversations covering multiple scam types. Thorough experimental evaluation shows that the ASR framework substantially enhances scam detection, particularly in challenging contexts such as job scams, and uncovers important demographic patterns in user vulnerability and perceptions of AI-generated assistance. Our findings reveal a contradiction where those most at risk are often least receptive to AI support, emphasizing the importance of user-centered design in AI-driven fraud prevention. This work advances both the practical and theoretical foundations for interpretable and human-centered AI systems in combating evolving digital threats.