Feedback-Guided Extraction of Knowledge Base from Retrieval-Augmented LLM Applications

TL;DR

This paper introduces CopyBreakRAG, a feedback-guided black-box attack that significantly improves the extraction of knowledge bases from retrieval-augmented LLM applications, highlighting security vulnerabilities.

Contribution

It presents a novel agent-based attack method that adaptively generates adversarial queries, surpassing prior approaches in extraction coverage under realistic black-box conditions.

Findings

Achieves 45% higher extraction coverage than previous methods.

Extracts over 70% of knowledge base data in commercial RAG applications.

Outperforms state-of-the-art black-box extraction techniques.

Abstract

Retrieval-Augmented Generation (RAG) expands the knowledge boundary of large language models (LLMs) by integrating external knowledge bases, whose construction is often time-consuming and laborious. If an adversary extracts the knowledge base verbatim, it not only severely infringes the owner's intellectual property but also enables the adversary to replicate the application's functionality for unfair competition. Previous works on knowledge base extraction are limited either by low extraction coverage (usually less than 4%) in query-based attacks or by impractical assumptions of white-box access in embedding-based optimization methods. In this work, we propose CopyBreakRAG, an agent-based black-box attack that reasons from feedback and adaptively generates new adversarial queries for progressive extraction. By balancing exploration and exploitation through curiosity-driven queries and feedback-guided query refinement, our method overcomes the limitations of prior approaches and achieves significantly higher extraction coverage in realistic black-box settings. Experimental results show that CopyBreakRAG outperforms the state-of-the-art black-box approach by 45% on average in terms of chunk extraction ratio from applications built with mainstream RAG frameworks, and extracts over 70% of the data from the knowledge base in applications on commercial platforms including OpenAI's GPTs and ByteDance's Coze when essential protection is in place.