Heterogeneous Federated Learning Systems for Time-Series Power Consumption Prediction with Multi-Head Embedding Mechanism

TL;DR

This paper introduces a novel federated learning system with multi-head embedding for accurate, privacy-preserving power consumption prediction in time-series data, outperforming existing methods.

Contribution

It proposes Multi-Head Heterogeneous Federated Learning (MHHFL) with embedded head networks and selection mechanisms, enhancing collaborative learning and prediction accuracy.

Findings

Reduces prediction error by up to 94.1%

Outperforms benchmark and state-of-the-art systems

Effective ablation studies confirm mechanism benefits

Abstract

Time-series prediction is increasingly popular in a variety of applications, such as smart factories and smart transportation. Researchers have used various techniques to predict power consumption, but existing models lack discussion of collaborative learning and privacy issues among multiple clients. To address these issues, we propose Multi-Head Heterogeneous Federated Learning (MHHFL) systems that consist of multiple head networks, which independently act as carriers for federated learning. In the federated period, each head network is embedded into 2-dimensional vectors and shared with the centralized source pool. MHHFL then selects appropriate source networks and blends the head networks as knowledge transfer in federated learning. The experimental results show that the proposed MHHFL systems significantly outperform the benchmark and state-of-the-art systems and reduce the prediction error by 24.9% to 94.1%. The ablation studies demonstrate the effectiveness of the proposed mechanisms in the MHHFL (head network embedding and selection mechanisms), which significantly outperforms traditional federated average and random transfer.