Towards Fair and Efficient De-identification: Quantifying the Efficiency and Generalizability of De-identification Approaches

TL;DR

This paper evaluates various transformer and large language models for clinical de-identification, demonstrating that smaller models can be both efficient and effective across multiple languages and cultural contexts, with the introduction of a new multi-cultural de-identification model.

Contribution

It systematically compares model sizes and languages for de-identification, and introduces BERT-MultiCulture-DEID to enhance robustness across cultures and languages.

Findings

Smaller models achieve comparable performance with lower inference costs.

Limited data fine-tuning enables smaller models to outperform larger ones in multilingual de-identification.

The new BERT-MultiCulture-DEID improves robustness across diverse languages and cultural contexts.

Abstract

Large language models (LLMs) have shown strong performance on clinical de-identification, the task of identifying sensitive identifiers to protect privacy. However, previous work has not examined their generalizability between formats, cultures, and genders. In this work, we systematically evaluate fine-tuned transformer models (BERT, ClinicalBERT, ModernBERT), small LLMs (Llama 1-8B, Qwen 1.5-7B), and large LLMs (Llama-70B, Qwen-72B) at de-identification. We show that smaller models achieve comparable performance while substantially reducing inference cost, making them more practical for deployment. Moreover, we demonstrate that smaller models can be fine-tuned with limited data to outperform larger models in de-identifying identifiers drawn from Mandarin, Hindi, Spanish, French, Bengali, and regional variations of English, in addition to gendered names. To improve robustness in multi-cultural contexts, we introduce and publicly release BERT-MultiCulture-DEID, a set of de-identification models based on BERT, ClinicalBERT, and ModernBERT, fine-tuned on MIMIC with identifiers from multiple language variants. Our findings provide the first comprehensive quantification of the efficiency-generalizability trade-off in de-identification and establish practical pathways for fair and efficient clinical de-identification. Details on accessing the models are available at: https://doi.org/10.5281/zenodo.18342291