Robust Federated Fine-Tuning in Heterogeneous Networks with Unreliable Connections: An Aggregation View

TL;DR

This paper introduces FedAuto, a robust federated fine-tuning framework that effectively handles unreliable network connections and data heterogeneity without prior network knowledge, ensuring convergence and improved performance.

Contribution

FedAuto is a novel adaptive aggregation method for federated fine-tuning that guarantees convergence under diverse network failures without requiring prior network condition knowledge.

Findings

FedAuto outperforms existing methods in unreliable network scenarios.

It guarantees convergence without assumptions on connection failure probabilities.

It is effective for both full-parameter and partial-parameter fine-tuning.

Abstract

Federated Fine-Tuning (FFT) has attracted growing interest as it leverages both server- and client-side data to enhance global model generalization while preserving privacy, and significantly reduces the computational burden on edge devices by avoiding training from scratch. Despite these advantages, FFT performance is often degraded by unreliable server-client connections and heterogeneous client data distributions. Most existing methods assume homogeneous network conditions or require prior knowledge of connection failures. However, these assumptions are impractical in real-world networks characterized by diverse communication standards (e.g., wired, Wi-Fi, 4G, and 5G) and heterogeneous failure patterns. To address these limitations, we propose FedAuto, a novel FFT framework that mitigates the combined effects of connection failures and data heterogeneity via adaptive aggregation. FedAuto operates without prior knowledge of network conditions or modifications to existing infrastructure, enabling seamless plug-and-play deployment. Moreover, we establish a rigorous convergence guarantee for FedAuto. By adopting a novel per-round aggregation perspective, our analysis removes the need for assumptions on connection failures probabilities or client selection strategies commonly imposed in prior work, and guarantees convergence of FedAuto for each individual realization, providing a stronger theoretical assurance. Extensive experiments demonstrate that FedAuto consistently outperforms state-of-the-art baselines under diverse connection failure scenarios for both full-parameter and partial-parameter fine-tuning (e.g., LoRA), and even surpasses strategies that rely on complex communication resource optimization.