MedAide: Leveraging Large Language Models for On-Premise Medical Assistance on Edge Devices

TL;DR

MedAide is an on-premise healthcare chatbot leveraging tiny-LLMs and model optimizations to provide efficient, privacy-preserving medical assistance on edge devices, achieving high accuracy and domain-specific performance.

Contribution

This work introduces MedAide, a novel edge-based medical assistant using tiny-LLMs with optimized training and deployment for resource-constrained environments.

Findings

Achieves 77% accuracy in medical consultations

Scores 56 on USMLE benchmark

Operates efficiently on consumer GPUs and Jetson boards

Abstract

Large language models (LLMs) are revolutionizing various domains with their remarkable natural language processing (NLP) abilities. However, deploying LLMs in resource-constrained edge computing and embedded systems presents significant challenges. Another challenge lies in delivering medical assistance in remote areas with limited healthcare facilities and infrastructure. To address this, we introduce MedAide, an on-premise healthcare chatbot. It leverages tiny-LLMs integrated with LangChain, providing efficient edge-based preliminary medical diagnostics and support. MedAide employs model optimizations for minimal memory footprint and latency on embedded edge devices without server infrastructure. The training process is optimized using low-rank adaptation (LoRA). Additionally, the model is trained on diverse medical datasets, employing reinforcement learning from human feedback (RLHF) to enhance its domain-specific capabilities. The system is implemented on various consumer GPUs and Nvidia Jetson development board. MedAide achieves 77\% accuracy in medical consultations and scores 56 in USMLE benchmark, enabling an energy-efficient healthcare assistance platform that alleviates privacy concerns due to edge-based deployment, thereby empowering the community.