eFedLLM: Efficient LLM Inference Based on Federated Learning

TL;DR

This paper presents eFedLLM, a federated learning approach that enhances the efficiency and accessibility of large language model inference through model parallelism, incentive mechanisms, and matrix optimization techniques.

Contribution

It introduces a novel federated learning framework with model parallelism and incentive mechanisms to improve LLM inference efficiency and security.

Findings

Significant reduction in computational and memory requirements.

Enhanced collaborative training of LLMs among resource-limited users.

Improved model robustness through incentive mechanisms.

Abstract

Large Language Models (LLMs) herald a transformative era in artificial intelligence (AI). However, the expansive scale of data and parameters of LLMs requires high-demand computational and memory resources, restricting their accessibility to a broader range of users and researchers. This paper introduces an effective approach that enhances the operational efficiency and affordability of LLM inference. By utilizing transformer-based federated learning (FL) with model-parallel distributed training, our model efficiently distributes the computational loads and memory requirements across a network of participants. This strategy permits users, especially those with limited resources to train state-of-the-art LLMs collaboratively. We also innovate an incentive mechanism within the FL framework, rewarding constructive contributions and filtering out malicious activities, thereby safeguarding the integrity and reliability of the training process. Concurrently, we leverage memory hierarchy strategies and Singular Value Decomposition (SVD) on weight matrices to boost computational and memory efficiencies further. Our results, derived from formulaic analyses and numerical calculations, demonstrate significant optimization of resource use and democratize access to cutting-edge LLMs, ensuring that a wide scale of users can both contribute to and benefit from these advanced models.