DoctorGLM: Fine-tuning your Chinese Doctor is not a Herculean Task

TL;DR

DoctorGLM is a cost-effective, Chinese medical dialogue model fine-tuned from ChatGLM-6B, aiming to improve healthcare AI accessibility and performance with a quick, affordable training process.

Contribution

This work demonstrates fine-tuning a large Chinese language model for medical dialogue tasks using accessible hardware, making healthcare AI development more feasible for hospitals.

Findings

Fine-tuned ChatGLM-6B on medical dialogues in 13 hours

Achieved affordable healthcare-specific LLM training

Shared initial model for community feedback

Abstract

The recent progress of large language models (LLMs), including ChatGPT and GPT-4, in comprehending and responding to human instructions has been remarkable. Nevertheless, these models typically perform better in English and have not been explicitly trained for the medical domain, resulting in suboptimal precision in diagnoses, drug recommendations, and other medical advice. Additionally, training and deploying a dialogue model is still believed to be impossible for hospitals, hindering the promotion of LLMs. To tackle these challenges, we have collected databases of medical dialogues in Chinese with ChatGPT's help and adopted several techniques to train an easy-deploy LLM. Remarkably, we were able to fine-tune the ChatGLM-6B on a single A100 80G in 13 hours, which means having a healthcare-purpose LLM can be very affordable. DoctorGLM is currently an early-stage engineering attempt and contain various mistakes. We are sharing it with the broader community to invite feedback and suggestions to improve its healthcare-focused capabilities: https://github.com/xionghonglin/DoctorGLM.