Automated Privacy Information Annotation in Large Language Model Interactions

TL;DR

This paper introduces a large-scale multilingual dataset and automated annotation pipeline for detecting private information leaks in user interactions with large language models, aiming to improve privacy protection.

Contribution

It presents a novel dataset and automated annotation method for privacy leak detection in LLM interactions, tailored for deployment on local devices.

Findings

Baseline models show performance gaps in real-world privacy detection.

The dataset enables comprehensive evaluation of privacy leakage detection methods.

Evaluation metrics cover privacy leakage, phrase extraction, and information accuracy.

Abstract

Users interacting with large language models (LLMs) under their real identifiers often unknowingly risk disclosing private information. Automatically notifying users whether their queries leak privacy and which phrases leak what private information has therefore become a practical need. Existing privacy detection methods, however, were designed for different objectives and application domains, typically tagging personally identifiable information (PII) in anonymous content, which is insufficient in real-name interaction scenarios with LLMs. In this work, to support the development and evaluation of privacy detection models for LLM interactions that are deployable on local user devices, we construct a large-scale multilingual dataset with 249K user queries and 154K annotated privacy phrases. In particular, we build an automated privacy annotation pipeline with strong LLMs to automatically extract privacy phrases from dialogue datasets and annotate leaked information. We also design evaluation metrics at the levels of privacy leakage, extracted privacy phrase, and privacy information. We further establish baseline methods using light-weight LLMs with both tuning-free and tuning-based methods, and report a comprehensive evaluation of their performance. Evaluation results reveal a gap between current performance and the requirements of real-world LLM applications, motivating future research into more effective local privacy detection methods grounded in our dataset.