Contrast with Major Classifier Vectors for Federated Medical Relation Extraction with Heterogeneous Label Distribution

TL;DR

This paper introduces FedCMC, a federated learning method for medical relation extraction that leverages major classifier vectors to improve model generalization across clients with heterogeneous label distributions, without data leakage.

Contribution

It proposes a novel ensemble-based classifier vector method, FedCMC, to enhance federated medical relation extraction under label heterogeneity.

Findings

FedCMC outperforms state-of-the-art FL algorithms on three datasets.

Requires minimal additional classifier parameter transfer.

Effectively mitigates overfitting to local label distributions.

Abstract

Federated medical relation extraction enables multiple clients to train a deep network collaboratively without sharing their raw medical data. In order to handle the heterogeneous label distribution across clients, most of the existing works only involve enforcing regularization between local and global models during optimization. In this paper, we fully utilize the models of all clients and propose a novel concept of \textit{major classifier vectors}, where a group of class vectors is obtained in an ensemble rather than the weighted average method on the server. The major classifier vectors are then distributed to all clients and the local training of each client is Contrasted with Major Classifier vectors (FedCMC), so the local model is not prone to overfitting to the local label distribution. FedCMC requires only a small amount of additional transfer of classifier parameters without any leakage of raw data, extracted representations, and label distributions. Our extensive experiments show that FedCMC outperforms the other state-of-the-art FL algorithms on three medical relation extraction datasets.