FraudShield: Knowledge Graph Empowered Defense for LLMs against Fraud Attacks

TL;DR

FraudShield is a novel framework that enhances LLM security against fraud by using a knowledge graph to identify and highlight suspicious content, improving detection and interpretability across various models and fraud types.

Contribution

This paper introduces FraudShield, a knowledge graph-based defense mechanism that improves LLM robustness against fraud through structured fraud tactic analysis and input augmentation.

Findings

Outperforms state-of-the-art defenses across multiple LLMs

Effective against five different fraud types

Provides interpretable evidence for model decisions

Abstract

Large language models (LLMs) have been widely integrated into critical automated workflows, including contract review and job application processes. However, LLMs are susceptible to manipulation by fraudulent information, which can lead to harmful outcomes. Although advanced defense methods have been developed to address this issue, they often exhibit limitations in effectiveness, interpretability, and generalizability, particularly when applied to LLM-based applications. To address these challenges, we introduce FraudShield, a novel framework designed to protect LLMs from fraudulent content by leveraging a comprehensive analysis of fraud tactics. Specifically, FraudShield constructs and refines a fraud tactic-keyword knowledge graph to capture high-confidence associations between suspicious text and fraud techniques. The structured knowledge graph augments the original input by highlighting keywords and providing supporting evidence, guiding the LLM toward more secure responses. Extensive experiments show that FraudShield consistently outperforms state-of-the-art defenses across four mainstream LLMs and five representative fraud types, while also offering interpretable clues for the model's generations.