Embedding with Large Language Models for Classification of HIPAA Safeguard Compliance Rules

TL;DR

This paper demonstrates that using multilingual BERT embeddings significantly improves the classification accuracy of HIPAA safeguard compliance rules in mHealth app code patterns, aiding developers in secure application development.

Contribution

It introduces the application of multilingual BERT embeddings for classifying HIPAA rules, outperforming traditional Word2Vec and existing methods in accuracy.

Findings

Logistic Regression achieved 99.95% accuracy.

SVM and Random Forest also achieved over 99% accuracy.

The approach outperforms existing classification methods.

Abstract

Although software developers of mHealth apps are responsible for protecting patient data and adhering to strict privacy and security requirements, many of them lack awareness of HIPAA regulations and struggle to distinguish between HIPAA rules categories. Therefore, providing guidance of HIPAA rules patterns classification is essential for developing secured applications for Google Play Store. In this work, we identified the limitations of traditional Word2Vec embeddings in processing code patterns. To address this, we adopt multilingual BERT (Bidirectional Encoder Representations from Transformers) which offers contextualized embeddings to the attributes of dataset to overcome the issues. Therefore, we applied this BERT to our dataset for embedding code patterns and then uses these embedded code to various machine learning approaches. Our results demonstrate that the models significantly enhances classification performance, with Logistic Regression achieving a remarkable accuracy of 99.95\%. Additionally, we obtained high accuracy from Support Vector Machine (99.79\%), Random Forest (99.73\%), and Naive Bayes (95.93\%), outperforming existing approaches. This work underscores the effectiveness and showcases its potential for secure application development.