FOCUS: Dealing with Label Quality Disparity in Federated Learning

TL;DR

This paper introduces FOCUS, a federated learning method that assesses client data credibility using benchmark samples to mitigate label noise and improve model performance in privacy-sensitive, heterogeneous environments.

Contribution

FOCUS proposes a novel credibility assessment technique for clients in federated learning, addressing label quality disparities without direct data access.

Findings

Effectively identifies clients with noisy labels.

Reduces impact of label noise on model performance.

Outperforms existing federated learning approaches.

Abstract

Ubiquitous systems with End-Edge-Cloud architecture are increasingly being used in healthcare applications. Federated Learning (FL) is highly useful for such applications, due to silo effect and privacy preserving. Existing FL approaches generally do not account for disparities in the quality of local data labels. However, the clients in ubiquitous systems tend to suffer from label noise due to varying skill-levels, biases or malicious tampering of the annotators. In this paper, we propose Federated Opportunistic Computing for Ubiquitous Systems (FOCUS) to address this challenge. It maintains a small set of benchmark samples on the FL server and quantifies the credibility of the client local data without directly observing them by computing the mutual cross-entropy between performance of the FL model on the local datasets and that of the client local FL model on the benchmark dataset. Then, a credit weighted orchestration is performed to adjust the weight assigned to clients in the FL model based on their credibility values. FOCUS has been experimentally evaluated on both synthetic data and real-world data. The results show that it effectively identifies clients with noisy labels and reduces their impact on the model performance, thereby significantly outperforming existing FL approaches.