Improving Convergence for Semi-Federated Learning: An Energy-Efficient Approach by Manipulating Over-the-Air Distortion

TL;DR

This paper introduces a semi-federated learning framework that strategically manipulates over-the-air distortion to accelerate convergence and reduce energy consumption, adapting the approach based on the stability region.

Contribution

It proposes a novel hybrid semi-federated learning method with over-the-air computation, including a distortion manipulation strategy and resource allocation algorithms for energy-efficient convergence.

Findings

Accelerates convergence by amplifying amplitude distortion in the non-stable region.

Maintains stability and improves final convergence by suppressing noise in the stable region.

Reduces energy consumption through optimized resource allocation algorithms.

Abstract

In this paper, we propose a hybrid learning framework that combines federated and split learning, termed semi-federated learning (SemiFL), in which over-the-air computation is utilized for gradient aggregation. A key idea is to strategically adjust the learning rate by manipulating over-the-air distortion for improving SemiFL's convergence. Specifically, we intentionally amplify amplitude distortion to increase the learning rate in the non-stable region, thereby accelerating convergence and reducing communication energy consumption. In the stable region, we suppress noise perturbation to maintain a small learning rate for improving SemiFL's final convergence. Theoretical results demonstrate the antagonistic effects of over-the-air distortion in different regions, under both independent and identically distributed (IID) and non-IID data settings. Then, we formulate two energy consumption minimization problems, one for each region, which implements a two-region mean square error threshold configuration scheme. Accordingly, we propose two resource allocation algorithms with closed-form solutions. Simulation results show that under different network and data distribution conditions, strategically manipulating over-the-air distortion can efficiently adjust the learning rate to improve SemiFL's convergence. Moreover, energy consumption can be reduced by using the proposed algorithms.