# The DNA dialect: a comprehensive guide to pretrained genomic language models

## TL;DR

This paper reviews genomic language models, their development, applications, and challenges in genomics.

## Contribution

It provides a comprehensive guide and analysis of genomic language models, highlighting trends and practical considerations.

## Key findings

- Genomic language models are increasing in complexity and diversity.
- Task-specific design and pretraining data are more impactful than general model scale.
- Applications show potential but also reveal unresolved gaps and pitfalls.

## Abstract

Following their success in natural language processing and protein biology, pretrained large language models have started appearing in genomics in large numbers. These genomic language models (gLMs), trained on diverse DNA and RNA sequences, promise improved performance on a variety of downstream prediction and understanding tasks. In this review, we trace the rapid evolution of gLMs, analyze current trends, and offer an overview of their application in genomic research. We investigate each gLM component in detail, from training data curation to the architecture, and highlight the present trends of increasing model complexity. We review major benchmarking efforts, suggesting that no single model dominates, and that task-specific design and pretraining data often outweigh general model scale or architecture. In addition, we discuss requirements for making gLMs practically useful for genomic research. While several applications, ranging from genome annotation to DNA sequence generation, showcase the potential of gLMs, their use highlights gaps and pitfalls that remain unresolved. This guide aims to equip researchers with a grounded understanding of gLM capabilities, limitations, and best practices for their effective use in genomics.

This Review provides a guide to help researchers gain a clear understanding of the capabilities, limitations, and best practices of genomic language models for their effective use in genomics.