Sensitive Data Detection with High-Throughput Machine Learning Models in Electrical Health Records

TL;DR

This paper presents a machine learning approach to automatically identify sensitive protected health information in electronic health records, enabling more efficient data sharing and de-identification across diverse healthcare datasets.

Contribution

It introduces a novel feature engineering method based on metadata differences and demonstrates high accuracy in detecting PHI fields across multiple datasets.

Findings

Achieved 99% accuracy in PHI detection on unseen datasets

Engineered over 30 features from metadata for classification

Effective across heterogeneous EHR data sources

Abstract

In the era of big data, there is an increasing need for healthcare providers, communities, and researchers to share data and collaborate to improve health outcomes, generate valuable insights, and advance research. The Health Insurance Portability and Accountability Act of 1996 (HIPAA) is a federal law designed to protect sensitive health information by defining regulations for protected health information (PHI). However, it does not provide efficient tools for detecting or removing PHI before data sharing. One of the challenges in this area of research is the heterogeneous nature of PHI fields in data across different parties. This variability makes rule-based sensitive variable identification systems that work on one database fail on another. To address this issue, our paper explores the use of machine learning algorithms to identify sensitive variables in structured data, thus facilitating the de-identification process. We made a key observation that the distributions of metadata of PHI fields and non-PHI fields are very different. Based on this novel finding, we engineered over 30 features from the metadata of the original features and used machine learning to build classification models to automatically identify PHI fields in structured Electronic Health Record (EHR) data. We trained the model on a variety of large EHR databases from different data sources and found that our algorithm achieves 99% accuracy when detecting PHI-related fields for unseen datasets. The implications of our study are significant and can benefit industries that handle sensitive data.