Uncertainty Minimization for Personalized Federated Semi-Supervised Learning

TL;DR

This paper introduces a novel personalized federated semi-supervised learning approach that leverages helper clients and an uncertainty-based metric to improve model performance and stability under data heterogeneity and limited labels.

Contribution

It proposes a new paradigm enabling partial-labeled clients to seek helper assistance and introduces an uncertainty-based helper selection method for robust federated learning.

Findings

Achieves superior performance over existing methods with partial labels.

Ensures stable convergence in highly heterogeneous data settings.

Reduces communication overhead through efficient helper selection.

Abstract

Since federated learning (FL) has been introduced as a decentralized learning technique with privacy preservation, statistical heterogeneity of distributed data stays the main obstacle to achieve robust performance and stable convergence in FL applications. Model personalization methods have been studied to overcome this problem. However, existing approaches are mainly under the prerequisite of fully labeled data, which is unrealistic in practice due to the requirement of expertise. The primary issue caused by partial-labeled condition is that, clients with deficient labeled data can suffer from unfair performance gain because they lack adequate insights of local distribution to customize the global model. To tackle this problem, 1) we propose a novel personalized semi-supervised learning paradigm which allows partial-labeled or unlabeled clients to seek labeling assistance from data-related clients (helper agents), thus to enhance their perception of local data; 2) based on this paradigm, we design an uncertainty-based data-relation metric to ensure that selected helpers can provide trustworthy pseudo labels instead of misleading the local training; 3) to mitigate the network overload introduced by helper searching, we further develop a helper selection protocol to achieve efficient communication with acceptable performance sacrifice. Experiments show that our proposed method can obtain superior performance and more stable convergence than other related works with partially labeled data, especially in highly heterogeneous setting.