BianCang: A Traditional Chinese Medicine Large Language Model

TL;DR

BianCang is a specialized large language model for traditional Chinese medicine, developed through a two-stage training process involving domain-specific knowledge injection and alignment, improving TCM diagnosis and syndrome differentiation.

Contribution

The paper introduces BianCang, a novel TCM-specific LLM trained with a comprehensive dataset and a two-stage process, addressing the gap in medical LLMs for traditional Chinese medicine.

Findings

BianCang outperforms 31 models across 11 test sets.

The model effectively enhances TCM diagnosis and syndrome differentiation.

Extensive datasets improve the understanding of TCM in LLMs.

Abstract

The surge of large language models (LLMs) has driven significant progress in medical applications, including traditional Chinese medicine (TCM). However, current medical LLMs struggle with TCM diagnosis and syndrome differentiation due to substantial differences between TCM and modern medical theory, and the scarcity of specialized, high-quality corpora. To this end, in this paper we propose BianCang, a TCM-specific LLM, using a two-stage training process that first injects domain-specific knowledge and then aligns it through targeted stimulation to enhance diagnostic and differentiation capabilities. Specifically, we constructed pre-training corpora, instruction-aligned datasets based on real hospital records, and the ChP-TCM dataset derived from the Pharmacopoeia of the People's Republic of China. We compiled extensive TCM and medical corpora for continual pre-training and supervised fine-tuning, building a comprehensive dataset to refine the model's understanding of TCM. Evaluations across 11 test sets involving 31 models and 4 tasks demonstrate the effectiveness of BianCang, offering valuable insights for future research. Code, datasets, and models are available on https://github.com/QLU-NLP/BianCang.