Exposing LLM Vulnerabilities: Adversarial Scam Detection and Performance

TL;DR

This paper examines the vulnerabilities of Large Language Models in scam detection by creating a detailed dataset with adversarial examples, revealing high misclassification rates and proposing strategies to enhance robustness.

Contribution

It introduces a comprehensive dataset with adversarial scam messages and analyzes LLM vulnerabilities, offering methods to improve their robustness against adversarial attacks.

Findings

Adversarial examples significantly increase misclassification rates.

A detailed dataset with nuanced scam types was created.

Proposed strategies improve LLM robustness against adversarial scams.

Abstract

Can we trust Large Language Models (LLMs) to accurately predict scam? This paper investigates the vulnerabilities of LLMs when facing adversarial scam messages for the task of scam detection. We addressed this issue by creating a comprehensive dataset with fine-grained labels of scam messages, including both original and adversarial scam messages. The dataset extended traditional binary classes for the scam detection task into more nuanced scam types. Our analysis showed how adversarial examples took advantage of vulnerabilities of a LLM, leading to high misclassification rate. We evaluated the performance of LLMs on these adversarial scam messages and proposed strategies to improve their robustness.