How global observation works in Federated Learning: Integrating vertical training into Horizontal Federated Learning

TL;DR

This paper introduces a vertical-horizontal federated learning (VHFL) framework that integrates global observations into horizontal FL, improving model accuracy while maintaining data security.

Contribution

The paper proposes a novel VHFL process that shares global features with agents without extra communication, enhancing accuracy and security in federated learning.

Findings

VHFL improves model accuracy over traditional horizontal FL.

The convergence of VHFL is analyzed considering delay and packet loss.

Experimental results validate the effectiveness of VHFL in real scenarios.

Abstract

Federated learning (FL) has recently emerged as a transformative paradigm that jointly train a model with distributed data sets in IoT while avoiding the need for central data collection. Due to the limited observation range, such data sets can only reflect local information, which limits the quality of trained models. In practice, the global information and local observations would require a joint consideration for learning to make a reasonable policy. However, in horizontal FL, the central agency only acts as a model aggregator without utilizing its global observation to further improve the model. This could significantly degrade the performance in some missions such as traffic flow prediction in network systems, where the global information may enhance the accuracy. Meanwhile, the global feature may not be directly transmitted to agents for data security. How to utilize the global observation residing in the central agency while protecting its safety thus rises up as an important problem in FL. In this paper, we develop a vertical-horizontal federated learning (VHFL) process, where the global feature is shared with the agents in a procedure similar to that of vertical FL without any extra communication rounds. By considering the delay and packet loss, we will analyze VHFL convergence and validate its performance by experiments. It is shown that the proposed VHFL could enhance the accuracy compared with horizontal FL while still protecting the security of global data.