SCRIPTMIND: Crime Script Inference and Cognitive Evaluation for LLM-based Social Engineering Scam Detection System

TL;DR

ScriptMind is a comprehensive framework that leverages LLMs and cognitive evaluation to improve social engineering scam detection and enhance user awareness through structured reasoning and real-time cognitive impact assessment.

Contribution

It introduces a novel integrated system combining scam reasoning, dataset creation, and cognitive evaluation to advance LLM-based scam detection and user cognition.

Findings

Fine-tuned small LLMs outperform GPT-4o in detection accuracy.

The system reduces false positives and improves scammer utterance prediction.

Cognitive simulation enhances user suspicion and awareness in scam scenarios.

Abstract

Social engineering scams increasingly employ personalized, multi-turn deception, exposing the limits of traditional detection methods. While Large Language Models (LLMs) show promise in identifying deception, their cognitive assistance potential remains underexplored. We propose ScriptMind, an integrated framework for LLM-based scam detection that bridges automated reasoning and human cognition. It comprises three components: the Crime Script Inference Task (CSIT) for scam reasoning, the Crime Script-Aware Inference Dataset (CSID) for fine-tuning small LLMs, and the Cognitive Simulation-based Evaluation of Social Engineering Defense (CSED) for assessing real-time cognitive impact. Using 571 Korean phone scam cases, we built 22,712 structured scammer-sequence training instances. Experimental results show that the 11B small LLM fine-tuned with ScriptMind outperformed GPT-4o by 13%, achieving superior performance over commercial models in detection accuracy, false-positive reduction, scammer utterance prediction, and rationale quality. Moreover, in phone scam simulation experiments, it significantly enhanced and sustained users' suspicion levels, improving their cognitive awareness of scams. ScriptMind represents a step toward human-centered, cognitively adaptive LLMs for scam defense.