Aloe: A Family of Fine-tuned Open Healthcare LLMs

TL;DR

Aloe introduces a family of open-source healthcare LLMs that are fine-tuned, aligned, and optimized with advanced inference techniques, achieving state-of-the-art performance and setting new standards for ethical and safe medical AI models.

Contribution

The paper presents Aloe, a new set of open healthcare LLMs trained on enhanced datasets, aligned with policy, and evaluated with comprehensive bias, toxicity, and risk assessments, along with improved inference strategies.

Findings

Aloe models achieve state-of-the-art results among open healthcare LLMs at 7B scale.

The models demonstrate improved ethical alignment and reduced bias and toxicity.

Advanced prompt engineering significantly boosts performance across benchmarks.

Abstract

As the capabilities of Large Language Models (LLMs) in healthcare and medicine continue to advance, there is a growing need for competitive open-source models that can safeguard public interest. With the increasing availability of highly competitive open base models, the impact of continued pre-training is increasingly uncertain. In this work, we explore the role of instruct tuning, model merging, alignment, red teaming and advanced inference schemes, as means to improve current open models. To that end, we introduce the Aloe family, a set of open medical LLMs highly competitive within its scale range. Aloe models are trained on the current best base models (Mistral, LLaMA 3), using a new custom dataset which combines public data sources improved with synthetic Chain of Thought (CoT). Aloe models undergo an alignment phase, becoming one of the first few policy-aligned open healthcare LLM using Direct Preference Optimization, setting a new standard for ethical performance in healthcare LLMs. Model evaluation expands to include various bias and toxicity datasets, a dedicated red teaming effort, and a much-needed risk assessment for healthcare LLMs. Finally, to explore the limits of current LLMs in inference, we study several advanced prompt engineering strategies to boost performance across benchmarks, yielding state-of-the-art results for open healthcare 7B LLMs, unprecedented at this scale.