Federated Learning Enhanced by Feature Reconstruction for Semantic Communication Module Updates of Agents

TL;DR

This paper introduces FedSFR, a federated learning framework with semantic feature reconstruction that improves model updates and communication efficiency in image semantic communication systems, especially under capacity constraints.

Contribution

FedSFR is the first to incorporate semantic feature reconstruction into federated learning for VQ-based image communication, enhancing stability and efficiency.

Findings

FedSFR outperforms existing methods on benchmark datasets.

It improves training stability and communication efficiency.

Effective in capacity-constrained environments.

Abstract

Recent advancements in semantic communication have primarily focused on image transmission, where neural network-based joint source-channel coding modules play a central role. However, such systems often experience semantic communication errors due to mismatched knowledge bases between agents and performance degradation from outdated models, necessitating regular model updates. To address these challenges in vector quantization (VQ)-based image semantic communication systems, we propose FedSFR, a novel federated learning framework that incorporates semantic feature reconstruction (FR). FedSFR introduces an FR step at the parameter server and allows a subset of clients to transmit compact feature vectors in lieu of sending full local model updates, thereby improving training stability and communication efficiency. To enable effective FR learning, we design a loss function tailored for VQ-based image semantic communication and demonstrate its validity as a surrogate for image reconstruction error. We further establish a rigorous convergence analysis of FedSFR. Experimental results on two benchmark datasets validate the superiority of FedSFR over existing baselines, especially in capacity-constrained settings, confirming both its effectiveness and robustness.