An In-Depth Evaluation of Federated Learning on Biomedical Natural Language Processing

TL;DR

This paper evaluates federated learning's effectiveness on biomedical NLP tasks, demonstrating its advantages in privacy-preserving collaborative training, resilience, and inference speed compared to traditional and large language models.

Contribution

It provides a comprehensive evaluation of federated learning on biomedical NLP, highlighting its performance benefits and robustness across multiple tasks and models.

Findings

FL models outperform individual client models

FL models sometimes match polled data models

Pre-trained transformers show resilience in FL settings

Abstract

Language models (LMs) such as BERT and GPT have revolutionized natural language processing (NLP). However, the medical field faces challenges in training LMs due to limited data access and privacy constraints imposed by regulations like the Health Insurance Portability and Accountability Act (HIPPA) and the General Data Protection Regulation (GDPR). Federated learning (FL) offers a decentralized solution that enables collaborative learning while ensuring data privacy. In this study, we evaluated FL on 2 biomedical NLP tasks encompassing 8 corpora using 6 LMs. Our results show that: 1) FL models consistently outperformed models trained on individual clients' data and sometimes performed comparably with models trained with polled data; 2) with the fixed number of total data, FL models training with more clients produced inferior performance but pre-trained transformer-based models exhibited great resilience. 3) FL models significantly outperformed large language models using zero-/one-shot learning and offered lightning inference speed.