Energy and Memory-Efficient Federated Learning With Ordered Layer Freezing

TL;DR

This paper introduces FedOLF, a federated learning method that improves efficiency by ordered layer freezing and Tensor Operation Approximation, achieving higher accuracy and lower resource consumption on IoT devices.

Contribution

FedOLF is the first to systematically apply ordered layer freezing combined with TOA for energy-efficient federated learning on resource-constrained IoT devices.

Findings

FedOLF outperforms existing methods in accuracy on multiple datasets.

It significantly reduces energy consumption and memory usage.

Experimental results show improved performance on non-iid data.

Abstract

Federated Learning (FL) has emerged as a privacy-preserving paradigm for training machine learning models across distributed edge devices in the Internet of Things (IoT). By keeping data local and coordinating model training through a central server, FL effectively addresses privacy concerns and reduces communication overhead. However, the limited computational power, memory, and bandwidth of IoT edge devices pose significant challenges to the efficiency and scalability of FL, especially when training deep neural networks. Various FL frameworks have been proposed to reduce computation and communication overheads through dropout or layer freezing. However, these approaches often sacrifice accuracy or neglect memory constraints. To this end, in this work, we introduce Federated Learning with Ordered Layer Freezing (FedOLF). FedOLF consistently freezes layers in a predefined order before training, significantly mitigating computation and memory requirements. To further reduce communication and energy costs, we incorporate Tensor Operation Approximation (TOA), a lightweight alternative to conventional quantization that better preserves model accuracy. Experimental results demonstrate that over non-iid data, FedOLF achieves at least 0.3%, 6.4%, 5.81%, 4.4%, 6.27% and 1.29% higher accuracy than existing works respectively on EMNIST (with CNN), CIFAR-10 (with AlexNet), CIFAR-100 (with ResNet20 and ResNet44), and CINIC-10 (with ResNet20 and ResNet44), along with higher energy efficiency and lower memory footprint.