AMSFL: Adaptive Multi-Step Federated Learning via Gradient Difference-Based Error Modeling

TL;DR

This paper introduces AMSFL, a federated learning framework that uses Gradient Difference Approximation to efficiently model errors and improve communication efficiency without sacrificing accuracy.

Contribution

It proposes GDA, a lightweight error estimation method based on first-order information, integrated into AMSFL for adaptive multi-step federated training.

Findings

GDA effectively estimates local errors with low computational cost

AMSFL improves communication efficiency in federated learning

The framework supports large-scale multi-step training environments

Abstract

Federated learning faces critical challenges in balancing communication efficiency and model accuracy. One key issue lies in the approximation of update errors without incurring high computational costs. In this paper, we propose a lightweight yet effective method called Gradient Difference Approximation (GDA), which leverages first-order information to estimate local error trends without computing the full Hessian matrix. The proposed method forms a key component of the Adaptive Multi-Step Federated Learning (AMSFL) framework and provides a unified error modeling strategy for large-scale multi-step adaptive training environments.