Multi-Site Clinical Federated Learning using Recursive and Attentive Models and NVFlare

TL;DR

This paper presents a federated learning framework using NVFlare to collaboratively train NLP models like LSTM and BERT on medical data, ensuring privacy and regulatory compliance while maintaining high accuracy.

Contribution

It introduces an integrated federated NLP framework with BERT pretraining tailored for medical data, addressing privacy concerns and demonstrating effective performance.

Findings

Effective federated training of BERT and LSTM models on medical data.

Enhanced privacy and compliance in multi-site medical NLP applications.

Maintained high accuracy comparable to centralized models.

Abstract

The prodigious growth of digital health data has precipitated a mounting interest in harnessing machine learning methodologies, such as natural language processing (NLP), to scrutinize medical records, clinical notes, and other text-based health information. Although NLP techniques have exhibited substantial potential in augmenting patient care and informing clinical decision-making, data privacy and adherence to regulations persist as critical concerns. Federated learning (FL) emerges as a viable solution, empowering multiple organizations to train machine learning models collaboratively without disseminating raw data. This paper proffers a pragmatic approach to medical NLP by amalgamating FL, NLP models, and the NVFlare framework, developed by NVIDIA. We introduce two exemplary NLP models, the Long-Short Term Memory (LSTM)-based model and Bidirectional Encoder Representations from Transformers (BERT), which have demonstrated exceptional performance in comprehending context and semantics within medical data. This paper encompasses the development of an integrated framework that addresses data privacy and regulatory compliance challenges while maintaining elevated accuracy and performance, incorporating BERT pretraining, and comprehensively substantiating the efficacy of the proposed approach.