KG-DF: A Black-box Defense Framework against Jailbreak Attacks Based on Knowledge Graphs

TL;DR

KG-DF is a novel defense framework utilizing knowledge graphs to detect and mitigate jailbreak attacks on large language models, balancing security and usability through structured semantic analysis.

Contribution

Introduces a knowledge graph-based, black-box defense framework with an extensible semantic parsing module to improve security against jailbreak attacks.

Findings

Enhanced defense performance against various jailbreak methods.

Improved response quality in general QA scenarios.

Effective identification of harmful inputs using knowledge graph associations.

Abstract

With the widespread application of large language models (LLMs) in various fields, the security challenges they face have become increasingly prominent, especially the issue of jailbreak. These attacks induce the model to generate erroneous or uncontrolled outputs through crafted inputs, threatening the generality and security of the model. Although existing defense methods have shown some effectiveness, they often struggle to strike a balance between model generality and security. Excessive defense may limit the normal use of the model, while insufficient defense may lead to security vulnerabilities. In response to this problem, we propose a Knowledge Graph Defense Framework (KG-DF). Specifically, because of its structured knowledge representation and semantic association capabilities, Knowledge Graph(KG) can be searched by associating input content with safe knowledge in the knowledge base, thus identifying potentially harmful intentions and providing safe reasoning paths. However, traditional KG methods encounter significant challenges in keyword extraction, particularly when confronted with diverse and evolving attack strategies. To address this issue, we introduce an extensible semantic parsing module, whose core task is to transform the input query into a set of structured and secure concept representations, thereby enhancing the relevance of the matching process. Experimental results show that our framework enhances defense performance against various jailbreak attack methods, while also improving the response quality of the LLM in general QA scenarios by incorporating domain-general knowledge.