Scientific Large Language Models: A Survey on Biological & Chemical Domains

TL;DR

This survey reviews recent advancements in scientific large language models specifically tailored for biological and chemical domains, highlighting their architectures, datasets, capabilities, challenges, and future research directions.

Contribution

It provides the first comprehensive, systematic overview of scientific LLMs in biology and chemistry, detailing technical developments and identifying key challenges and opportunities.

Findings

Analyzed various model architectures and datasets used in scientific LLMs.

Identified key challenges such as data scarcity and model interpretability.

Outlined promising future research directions in scientific LLM development.

Abstract

Large Language Models (LLMs) have emerged as a transformative power in enhancing natural language comprehension, representing a significant stride toward artificial general intelligence. The application of LLMs extends beyond conventional linguistic boundaries, encompassing specialized linguistic systems developed within various scientific disciplines. This growing interest has led to the advent of scientific LLMs, a novel subclass specifically engineered for facilitating scientific discovery. As a burgeoning area in the community of AI for Science, scientific LLMs warrant comprehensive exploration. However, a systematic and up-to-date survey introducing them is currently lacking. In this paper, we endeavor to methodically delineate the concept of "scientific language", whilst providing a thorough review of the latest advancements in scientific LLMs. Given the expansive realm of scientific disciplines, our analysis adopts a focused lens, concentrating on the biological and chemical domains. This includes an in-depth examination of LLMs for textual knowledge, small molecules, macromolecular proteins, genomic sequences, and their combinations, analyzing them in terms of model architectures, capabilities, datasets, and evaluation. Finally, we critically examine the prevailing challenges and point out promising research directions along with the advances of LLMs. By offering a comprehensive overview of technical developments in this field, this survey aspires to be an invaluable resource for researchers navigating the intricate landscape of scientific LLMs.