CAPID: Context-Aware PII Detection for Question-Answering Systems

TL;DR

This paper introduces CAPID, a privacy-preserving system that fine-tunes small language models to detect and classify contextually relevant PII in user queries, improving privacy and response quality in question-answering systems.

Contribution

It presents a novel synthetic data generation pipeline and a fine-tuning approach for small language models to detect and classify relevant PII in context.

Findings

Relevance-aware PII detection outperforms existing baselines.

The approach preserves higher downstream utility.

The synthetic dataset covers diverse PII types and relevance levels.

Abstract

Detecting personally identifiable information (PII) in user queries is critical for ensuring privacy in question-answering systems. Current approaches mainly redact all PII, disregarding the fact that some of them may be contextually relevant to the user's question, resulting in a degradation of response quality. Large language models (LLMs) might be able to help determine which PII are relevant, but due to their closed source nature and lack of privacy guarantees, they are unsuitable for sensitive data processing. To achieve privacy-preserving PII detection, we propose CAPID, a practical approach that fine-tunes a locally owned small language model (SLM) that filters sensitive information before it is passed to LLMs for QA. However, existing datasets do not capture the context-dependent relevance of PII needed to train such a model effectively. To fill this gap, we propose a synthetic data generation pipeline that leverages LLMs to produce a diverse, domain-rich dataset spanning multiple PII types and relevance levels. Using this dataset, we fine-tune an SLM to detect PII spans, classify their types, and estimate contextual relevance. Our experiments show that relevance-aware PII detection with a fine-tuned SLM substantially outperforms existing baselines in span, relevance and type accuracy while preserving significantly higher downstream utility under anonymization.