Unlocking the Potential of Large Language Models for Clinical Text Anonymization: A Comparative Study

TL;DR

This study evaluates the effectiveness of large language models in clinical text anonymization, proposing new metrics and demonstrating their reliability compared to traditional methods for safeguarding patient privacy.

Contribution

The paper introduces six new evaluation metrics and provides a comparative analysis of LLM-based anonymization methods versus baseline techniques.

Findings

LLM-based models outperform traditional methods in anonymization tasks.

New evaluation metrics better capture the challenges of generative anonymization.

LLMs show promise as reliable tools for clinical text privacy protection.

Abstract

Automated clinical text anonymization has the potential to unlock the widespread sharing of textual health data for secondary usage while assuring patient privacy and safety. Despite the proposal of many complex and theoretically successful anonymization solutions in literature, these techniques remain flawed. As such, clinical institutions are still reluctant to apply them for open access to their data. Recent advances in developing Large Language Models (LLMs) pose a promising opportunity to further the field, given their capability to perform various tasks. This paper proposes six new evaluation metrics tailored to the challenges of generative anonymization with LLMs. Moreover, we present a comparative study of LLM-based methods, testing them against two baseline techniques. Our results establish LLM-based models as a reliable alternative to common approaches, paving the way toward trustworthy anonymization of clinical text.