Knowledge-Aware Federated Active Learning with Non-IID Data

TL;DR

This paper introduces KAFAL, a federated active learning framework that effectively selects informative data and compensates for client heterogeneity in non-IID data settings, reducing annotation costs.

Contribution

The paper proposes Knowledge-Aware Federated Active Learning (KAFAL), combining novel sampling and update strategies to improve federated learning with limited labels and non-IID data.

Findings

KSAS outperforms existing active learning methods.

KCFU enhances client model performance under data heterogeneity.

KAFAL reduces annotation costs while maintaining high accuracy.

Abstract

Federated learning enables multiple decentralized clients to learn collaboratively without sharing the local training data. However, the expensive annotation cost to acquire data labels on local clients remains an obstacle in utilizing local data. In this paper, we propose a federated active learning paradigm to efficiently learn a global model with limited annotation budget while protecting data privacy in a decentralized learning way. The main challenge faced by federated active learning is the mismatch between the active sampling goal of the global model on the server and that of the asynchronous local clients. This becomes even more significant when data is distributed non-IID across local clients. To address the aforementioned challenge, we propose Knowledge-Aware Federated Active Learning (KAFAL), which consists of Knowledge-Specialized Active Sampling (KSAS) and Knowledge-Compensatory Federated Update (KCFU). KSAS is a novel active sampling method tailored for the federated active learning problem. It deals with the mismatch challenge by sampling actively based on the discrepancies between local and global models. KSAS intensifies specialized knowledge in local clients, ensuring the sampled data to be informative for both the local clients and the global model. KCFU, in the meantime, deals with the client heterogeneity caused by limited data and non-IID data distributions. It compensates for each client's ability in weak classes by the assistance of the global model. Extensive experiments and analyses are conducted to show the superiority of KSAS over the state-of-the-art active learning methods and the efficiency of KCFU under the federated active learning framework.