Genomic Language Models: Opportunities and Challenges

TL;DR

Genomic Language Models (gLMs) are large language models trained on DNA sequences that hold promise for advancing genomic understanding, but face significant development challenges, especially for complex genomes.

Contribution

This paper reviews the opportunities, applications, and challenges of developing effective gLMs for biological sequence analysis.

Findings

gLMs can predict functional constraints and aid in sequence design

Transfer learning enhances gLM applications in genomics

Developing gLMs for complex genomes remains challenging

Abstract

Large language models (LLMs) are having transformative impacts across a wide range of scientific fields, particularly in the biomedical sciences. Just as the goal of Natural Language Processing is to understand sequences of words, a major objective in biology is to understand biological sequences. Genomic Language Models (gLMs), which are LLMs trained on DNA sequences, have the potential to significantly advance our understanding of genomes and how DNA elements at various scales interact to give rise to complex functions. To showcase this potential, we highlight key applications of gLMs, including functional constraint prediction, sequence design, and transfer learning. Despite notable recent progress, however, developing effective and efficient gLMs presents numerous challenges, especially for species with large, complex genomes. Here, we discuss major considerations for developing and evaluating gLMs.