Advancing bioinformatics with large language models: components, applications and perspectives

TL;DR

This review explores how large language models are transforming bioinformatics by detailing their components, applications across various biological data types, and offering practical guidance for their effective use and development.

Contribution

It provides a comprehensive overview of LLM components, applications in bioinformatics, and practical strategies, highlighting their potential beyond natural language processing.

Findings

LLMs are effective in genomics, transcriptomics, proteomics, drug discovery, and single-cell analysis.

Current foundation models enable diverse bioinformatics applications.

Guidelines are provided for optimizing LLM use and fostering innovation.

Abstract

Large language models (LLMs) are a class of artificial intelligence models based on deep learning, which have great performance in various tasks, especially in natural language processing (NLP). Large language models typically consist of artificial neural networks with numerous parameters, trained on large amounts of unlabeled input using self-supervised or semi-supervised learning. However, their potential for solving bioinformatics problems may even exceed their proficiency in modeling human language. In this review, we will provide a comprehensive overview of the essential components of large language models (LLMs) in bioinformatics, spanning genomics, transcriptomics, proteomics, drug discovery, and single-cell analysis. Key aspects covered include tokenization methods for diverse data types, the architecture of transformer models, the core attention mechanism, and the pre-training processes underlying these models. Additionally, we will introduce currently available foundation models and highlight their downstream applications across various bioinformatics domains. Finally, drawing from our experience, we will offer practical guidance for both LLM users and developers, emphasizing strategies to optimize their use and foster further innovation in the field.