ScamPilot: Simulating Conversations with LLMs to Protect Against Online Scams

TL;DR

ScamPilot is a conversational tool using simulated scam interactions with LLMs to educate users on recognizing and defending against online scams, improving scam detection and user confidence.

Contribution

This paper introduces ScamPilot, a novel interactive system that uses LLM-powered agents to simulate scams and train users through real-time feedback and dynamic interaction.

Findings

8% increase in scam recognition

19% increase in self-efficacy

Users provided more action-oriented advice

Abstract

Fraud continues to proliferate online, from phishing and ransomware to impersonation scams. Yet automated prevention approaches adapt slowly and may not reliably protect users from falling prey to new scams. To better combat online scams, we developed ScamPilot, a conversational interface that inoculates users against scams through simulation, dynamic interaction, and real-time feedback. ScamPilot simulates scams with two large language model-powered agents: a scammer and a target. Users must help the target defend against the scammer by providing real-time advice. Through a between-subjects study (N=150) with one control and three experimental conditions, we find that blending advice-giving with multiple choice questions significantly increased scam recognition (+8%) without decreasing wariness towards legitimate conversations. Users' response efficacy and change in self-efficacy was also 9% and 19% higher, respectively. Qualitatively, we find that users more frequently provided action-oriented advice over urging caution or providing emotional support. Overall, ScamPilot demonstrates the potential for inter-agent conversational user interfaces to augment learning.