Federated Learning with Only Positive Labels by Exploring Label Correlations

TL;DR

This paper introduces FedALC, a federated learning method that leverages label correlations to improve multi-label classification with positive-only data, enhancing performance while reducing communication costs.

Contribution

The paper proposes a novel federated learning approach that estimates label correlations for better multi-label classification with positive-only data, and introduces a fixed embedding variant for efficiency and privacy.

Findings

FedALC significantly outperforms existing methods on multiple datasets.

The fixed embedding variant reduces communication overhead and enhances privacy.

Leveraging label correlations improves model performance in positive-only federated learning.

Abstract

Federated learning aims to collaboratively learn a model by using the data from multiple users under privacy constraints. In this paper, we study the multi-label classification problem under the federated learning setting, where trivial solution and extremely poor performance may be obtained, especially when only positive data w.r.t. a single class label are provided for each client. This issue can be addressed by adding a specially designed regularizer on the server-side. Although effective sometimes, the label correlations are simply ignored and thus sub-optimal performance may be obtained. Besides, it is expensive and unsafe to exchange user's private embeddings between server and clients frequently, especially when training model in the contrastive way. To remedy these drawbacks, we propose a novel and generic method termed Federated Averaging by exploring Label Correlations (FedALC). Specifically, FedALC estimates the label correlations in the class embedding learning for different label pairs and utilizes it to improve the model training. To further improve the safety and also reduce the communication overhead, we propose a variant to learn fixed class embedding for each client, so that the server and clients only need to exchange class embeddings once. Extensive experiments on multiple popular datasets demonstrate that our FedALC can significantly outperform existing counterparts.