LlamaCare: A Large Medical Language Model for Enhancing Healthcare Knowledge Sharing

TL;DR

LlamaCare is a fine-tuned medical language model with an innovative classification module, achieving high performance on healthcare benchmarks while being environmentally friendly and openly available.

Contribution

The paper introduces LlamaCare, a low-carbon, high-performance medical language model with a novel classification module, extending capabilities in healthcare knowledge sharing.

Findings

Achieved performance comparable to ChatGPT on medical benchmarks

Reduced carbon emissions during model fine-tuning

Released datasets for one-shot and few-shot learning

Abstract

Large language models (LLMs) have shown amazing capabilities in knowledge memorization and the present. However, when it comes to domain-specific knowledge and downstream tasks like medical, general LLMs are often unable to give precise answers. In addition, when people want LLMs to answer classification questions, they usually go through instruction tuning first. However, LLMs do not always give a direct index of the categorization after instruction tuning. In this paper, we proposed LlamaCare, a fine-tuned medical language model, and Extended Classification Integration(ECI), a module to handle classification problems of LLMs. Our contributions are : (i) We fine-tuned a large language model of medical knowledge with very low carbon emissions and achieved similar performance with ChatGPT by a 24G GPU. (ii) We solved the problem of redundant categorical answers and improved the performance of LLMs by proposing a new module called Extended Classification Integration. (iii) We released our processed data for one-shot and few-shot training for some benchmarks such as PubMedQA and USMLE 1-3 step. Our method achieves a close performance comparable to some state-of-the-art models with the same quantity of parameters on benchmarks, while being more environmentally friendly by using less GPU computation time. Our models, codes, and datasets can be found at \url{https://github.com/Stephen-SMJ/LLamaCare}.