# Automatic end-to-end De-identification: Is high accuracy the only metric?

## TL;DR

This paper reviews the progress and challenges in automatic end-to-end de-identification of electronic health records, emphasizing that high accuracy alone is insufficient for ensuring patient privacy and data utility.

## Contribution

It provides the first comprehensive review of end-to-end automatic de-identification systems, highlighting recent advancements and ongoing challenges beyond accuracy.

## Key findings

- High accuracy systems have been developed for PHI identification.
- Challenges remain in surrogate generation and PHI replacement.
- Risks to patient privacy persist despite high accuracy.

## Abstract

De-identification of electronic health records (EHR) is a vital step towards advancing health informatics research and maximising the use of available data. It is a two-step process where step one is the identification of protected health information (PHI), and step two is replacing such PHI with surrogates. Despite the recent advances in automatic de-identification of EHR, significant obstacles remain if the abundant health data available are to be used to the full potential. Accuracy in de-identification could be considered a necessary, but not sufficient condition for the use of EHR without individual patient consent. We present here a comprehensive review of the progress to date, both the impressive successes in achieving high accuracy and the significant risks and challenges that remain. To best of our knowledge, this is the first paper to present a complete picture of end-to-end automatic de-identification. We review 18 recently published automatic de-identification systems -designed to de-identify EHR in the form of free text- to show the advancements made in improving the overall accuracy of the system, and in identifying individual PHI. We argue that despite the improvements in accuracy there remain challenges in surrogate generation and replacements of identified PHIs, and the risks posed to patient protection and privacy.