Thunder-DeID: Accurate and Efficient De-identification Framework for Korean Court Judgments

TL;DR

Thunder-DeID is a novel deep learning framework designed to efficiently and accurately de-identify Korean court judgments, balancing legal compliance with data privacy and open access.

Contribution

It introduces the first Korean legal dataset for de-identification, a systematic PII categorization, and an end-to-end neural network pipeline for legal text de-identification.

Findings

Achieves state-of-the-art de-identification performance

Provides a comprehensive Korean legal PII dataset

Demonstrates effectiveness of neural network approach in legal texts

Abstract

To ensure a balance between open access to justice and personal data protection, the South Korean judiciary mandates the de-identification of court judgments before they can be publicly disclosed. However, the current de-identification process is inadequate for handling court judgments at scale while adhering to strict legal requirements. Additionally, the legal definitions and categorizations of personal identifiers are vague and not well-suited for technical solutions. To tackle these challenges, we propose a de-identification framework called Thunder-DeID, which aligns with relevant laws and practices. Specifically, we (i) construct and release the first Korean legal dataset containing annotated judgments along with corresponding lists of entity mentions, (ii) introduce a systematic categorization of Personally Identifiable Information (PII), and (iii) develop an end-to-end deep neural network (DNN)-based de-identification pipeline. Our experimental results demonstrate that our model achieves state-of-the-art performance in the de-identification of court judgments.