PharmaGPT: Domain-Specific Large Language Models for Bio-Pharmaceutical and Chemistry

TL;DR

PharmaGPT introduces domain-specific large language models with 13B and 70B parameters, trained on bio-pharmaceutical and chemical data, outperforming general models on specialized benchmarks with fewer parameters.

Contribution

The paper presents PharmaGPT, a new suite of large language models tailored for bio-pharmaceutical and chemical domains, demonstrating superior performance with fewer parameters than general-purpose models.

Findings

PharmaGPT outperforms existing models on domain-specific benchmarks.

Fewer parameters are needed for high performance in specialized tasks.

The models set new standards for NLP in bio-pharmaceutical and chemical fields.

Abstract

Large language models (LLMs) have revolutionized Natural Language Processing (NLP) by minimizing the need for complex feature engineering. However, the application of LLMs in specialized domains like biopharmaceuticals and chemistry remains largely unexplored. These fields are characterized by intricate terminologies, specialized knowledge, and a high demand for precision areas where general purpose LLMs often fall short. In this study, we introduce PharmaGPT, a suite of domain specilized LLMs with 13 billion and 70 billion parameters, specifically trained on a comprehensive corpus tailored to the Bio-Pharmaceutical and Chemical domains. Our evaluation shows that PharmaGPT surpasses existing general models on specific-domain benchmarks such as NAPLEX, demonstrating its exceptional capability in domain-specific tasks. Remarkably, this performance is achieved with a model that has only a fraction, sometimes just one-tenth-of the parameters of general-purpose large models. This advancement establishes a new benchmark for LLMs in the bio-pharmaceutical and chemical fields, addressing the existing gap in specialized language modeling. It also suggests a promising path for enhanced research and development, paving the way for more precise and effective NLP applications in these areas.