Election of Collaborators via Reinforcement Learning for Federated Brain Tumor Segmentation

TL;DR

This paper introduces RL-HSimAgg, a reinforcement learning-based method for selecting collaborators in federated brain tumor segmentation, improving model performance and robustness through multi-armed bandit algorithms like UCB.

Contribution

It proposes a novel RL-based collaborator selection framework using multi-armed bandit algorithms to enhance federated learning in medical image segmentation.

Findings

UCB algorithm outperforms Epsilon-greedy in Dice scores

RL-based collaborator selection improves segmentation accuracy

UCB enhances model robustness in federated brain tumor segmentation

Abstract

Federated learning (FL) enables collaborative model training across decentralized datasets while preserving data privacy. However, optimally selecting participating collaborators in dynamic FL environments remains challenging. We present RL-HSimAgg, a novel reinforcement learning (RL) and similarity-weighted aggregation (simAgg) algorithm using harmonic mean to manage outlier data points. This paper proposes applying multi-armed bandit algorithms to improve collaborator selection and model generalization. By balancing exploration-exploitation trade-offs, these RL methods can promote resource-efficient training with diverse datasets. We demonstrate the effectiveness of Epsilon-greedy (EG) and upper confidence bound (UCB) algorithms for federated brain lesion segmentation. In simulation experiments on internal and external validation sets, RL-HSimAgg with UCB collaborator outperformed the EG method across all metrics, achieving higher Dice scores for Enhancing Tumor (0.7334 vs 0.6797), Tumor Core (0.7432 vs 0.6821), and Whole Tumor (0.8252 vs 0.7931) segmentation. Therefore, for the Federated Tumor Segmentation Challenge (FeTS 2024), we consider UCB as our primary client selection approach in federated Glioblastoma lesion segmentation of multi-modal MRIs. In conclusion, our research demonstrates that RL-based collaborator management, e.g. using UCB, can potentially improve model robustness and flexibility in distributed learning environments, particularly in domains like brain tumor segmentation.