ALDEN: Boosting Private Data Extraction from Retrieval-Augmented Generation Systems via Active Learning and Distribution Estimation

TL;DR

ALDEN is a new attack method that significantly improves private data extraction from RAG systems by using active learning and distribution estimation, highlighting vulnerabilities in current models.

Contribution

It introduces ALDEN, a novel attack combining active learning and distribution estimation to enhance data extraction efficiency from RAG systems.

Findings

ALDEN outperforms existing methods in data extraction rates.

Active learning diversifies malicious queries effectively.

Distribution estimation guides query generation for better results.

Abstract

Retrieval-Augmented Generation (RAG) is widely used to augment large language models with external knowledge retrieval to improve reliability and generalization. However, recent studies have shown that RAG systems remain vulnerable to data extraction attacks, where adversaries can extract private data by embedding malicious commands into user queries. Despite their feasibility, existing attacks typically suffer from low data extraction rates and limited practical effectiveness. Here, we propose ALDEN, a novel attack that effectively and efficiently extracts private data from RAGs. First, we employ active learning to diversify malicious queries and improve data extraction rates. Second, we observe that the data distribution of the underlying knowledge base provides valuable guidance for query generation and introduce a decay-based dynamic algorithm to estimate the corresponding topic distribution. By combining them together, we demonstrate that ALDEN substantially outperforms state-of-the-art methods through comprehensive evaluations.