PsyScam: A Benchmark for Psychological Techniques in Real-World Scams
Shang Ma, Tianyi Ma, Jiahao Liu, Wei Song, Zhenkai Liang, Xusheng Xiao, Yanfang Ye

TL;DR
PsyScam is a benchmark dataset that captures psychological techniques used in real-world scams and evaluates how well models can detect and generate scam content leveraging these techniques, highlighting challenges for current AI systems.
Contribution
Introduces PsyScam, a comprehensive benchmark dataset with annotations of psychological techniques in scams, and explores LLMs' ability to generate scam variants based on these techniques.
Findings
Existing models struggle to detect scams using PTs.
LLMs can generate diverse scam variants based on PTs.
PsyScam challenges current AI defenses against scams.
Abstract
Over the years, online scams have grown dramatically, with nearly 50% of global consumers encountering scam attempts each week. These scams cause not only significant financial losses to individuals and businesses, but also lasting psychological trauma, largely due to scammers' strategic employment of psychological techniques (PTs) to manipulate victims. Meanwhile, scammers continually evolve their tactics by leveraging advances in Large Language Models (LLMs) to generate diverse scam variants that easily bypass existing defenses. To address this pressing problem, we introduce PsyScam, a benchmark designed to systematically capture the PTs employed in real-world scam reports, and investigate how LLMs can be utilized to generate variants of scams based on the PTs and the contexts provided by these scams. Specifically, we collect a wide range of scam reports and ground its annotations…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsBlockchain Technology Applications and Security · Digital Mental Health Interventions
