Hippocrates: An Open-Source Framework for Advancing Large Language Models in Healthcare

TL;DR

Hippocrates is an open-source framework that provides unrestricted access to medical LLM resources, fostering collaborative research and surpassing existing models in performance to accelerate healthcare AI advancements.

Contribution

It introduces Hippocrates, an open-source medical LLM framework with transparent resources and a family of high-performing models, promoting reproducibility and innovation in healthcare AI.

Findings

Hippocrates offers unrestricted access to datasets, code, and evaluation protocols.

Hippo models outperform existing open medical LLMs significantly.

Models surpass larger proprietary models in medical NLP tasks.

Abstract

The integration of Large Language Models (LLMs) into healthcare promises to transform medical diagnostics, research, and patient care. Yet, the progression of medical LLMs faces obstacles such as complex training requirements, rigorous evaluation demands, and the dominance of proprietary models that restrict academic exploration. Transparent, comprehensive access to LLM resources is essential for advancing the field, fostering reproducibility, and encouraging innovation in healthcare AI. We present Hippocrates, an open-source LLM framework specifically developed for the medical domain. In stark contrast to previous efforts, it offers unrestricted access to its training datasets, codebase, checkpoints, and evaluation protocols. This open approach is designed to stimulate collaborative research, allowing the community to build upon, refine, and rigorously evaluate medical LLMs within a transparent ecosystem. Also, we introduce Hippo, a family of 7B models tailored for the medical domain, fine-tuned from Mistral and LLaMA2 through continual pre-training, instruction tuning, and reinforcement learning from human and AI feedback. Our models outperform existing open medical LLMs models by a large-margin, even surpassing models with 70B parameters. Through Hippocrates, we aspire to unlock the full potential of LLMs not just to advance medical knowledge and patient care but also to democratize the benefits of AI research in healthcare, making them available across the globe.