FedCoT: Communication-Efficient Federated Reasoning Enhancement for Large Language Models

TL;DR

FedCoT is a federated learning framework that enhances reasoning and interpretability in large language models, especially for medical applications, by using lightweight chain-of-thought mechanisms and noise-free aggregation to improve performance under privacy and resource constraints.

Contribution

This paper introduces FedCoT, a novel federated reasoning enhancement method that improves LLM reasoning accuracy, interpretability, and robustness while reducing communication overhead and preserving privacy.

Findings

Significantly improves reasoning accuracy in medical tasks.

Reduces communication overhead compared to traditional federated fine-tuning.

Maintains data privacy and client heterogeneity handling.

Abstract

Efficiently enhancing the reasoning capabilities of large language models (LLMs) in federated learning environments remains challenging, particularly when balancing performance gains with strict computational, communication, and privacy constraints. This challenge is especially acute in healthcare, where decisions-spanning clinical, operational, and patient-facing contexts-demand not only accurate outputs but also interpretable, traceable rationales to ensure safety, accountability, and regulatory compliance. Conventional federated tuning approaches on LLM fail to address this need: they optimize primarily for answer correctness while neglecting rationale quality, leaving CoT capabilities dependent on models' innate pre-training abilities. Moreover, existing methods for improving rationales typically rely on privacy-violating knowledge distillation from centralized models. Additionally, the communication overhead in traditional federated fine-tuning on LLMs remains substantial. We addresses this gap by proposing FedCoT, a novel framework specifically designed to enhance reasoning in federated settings. FedCoT leverages a lightweight chain-of-thought enhancement mechanism: local models generate multiple reasoning paths, and a compact discriminator dynamically selects the most promising one. This approach improves reasoning accuracy and robustness while providing valuable interpretability, which is particularly critical for medical applications. To manage client heterogeneity efficiently, we adopt an improved aggregation approach building upon advanced LoRA module stacking, incorporating client classifier-awareness to achieve noise-free aggregation across diverse clients. Comprehensive experiments on medical reasoning tasks demonstrate that FedCoT significantly boosts client-side reasoning performance under stringent resource budgets while fully preserving data privacy.