Generalization of Medical Large Language Models through Cross-Domain Weak Supervision

TL;DR

This paper introduces ICFT, a novel fine-tuning framework that significantly improves the domain-specific capabilities of medical large language models by combining curriculum learning, memory coordination, and parameter efficiency, leading to better performance and generalization.

Contribution

The paper presents a new ICFT framework that enhances medical LLMs through incremental, curriculum-based training with memory and parameter-efficient methods, outperforming existing approaches.

Findings

ICFT improves accuracy across diverse medical NLP tasks.

ICFT enhances generalization to unseen medical data.

ICFT reduces errors and increases response diversity.

Abstract

The advancement of large language models (LLMs) has opened new frontiers in natural language processing, particularly in specialized domains like healthcare. In this paper, we propose the Incremental Curriculum-Based Fine-Tuning (ICFT) framework to enhance the generative capabilities of medical large language models (MLLMs). ICFT combines curriculum-based learning, dual-stage memory coordination, and parameter-efficient fine-tuning to enable a progressive transition from general linguistic knowledge to strong domain-specific expertise. Experimental results across diverse medical NLP tasks, including question answering, preference classification, and response generation, demonstrate that ICFT consistently outperforms state-of-the-art baselines, achieving improvements in both accuracy and efficiency. Further analysis reveals the framework's ability to generalize to unseen data, reduce errors, and deliver diverse, contextually relevant medical responses. These findings establish ICFT as a robust and scalable solution for adapting LLMs to the medical domain, offering practical benefits for real-world healthcare applications.