# Embeddings from language models are good learners for single-cell data analysis

## TL;DR

scELMo uses large language models to analyze single-cell data efficiently, enabling tasks like cell clustering and treatment prediction with less computational power.

## Contribution

scELMo introduces a novel framework that combines pre-trained language models with single-cell data analysis for improved performance and accessibility.

## Key findings

- scELMo achieves cell clustering, batch effect correction, and cell-type annotation without training a new model.
- The method supports in silico treatment analysis and perturbation modeling through a fine-tuning framework.
- scELMo requires fewer resources and offers a scalable, interpretable solution for single-cell data analysis.

## Abstract

Foundation models (FMs) have been built to analyze single-cell data with different degrees of success. Here, we present scELMo (single-cell embedding from language models), a method for analyzing single-cell data with the help of large language models (LLMs). LLMs can generate both the description of metadata information and the embeddings for such descriptions. We then combine the embeddings from LLMs with the raw data under the zero-shot learning framework to further extend its function by using the fine-tuning framework to handle different tasks. We demonstrate that scELMo is capable of cell clustering, batch effect correction, and cell-type annotation without training a new model. Moreover, the fine-tuning framework of scELMo can help with more challenging tasks, including in silico treatment analysis or modeling perturbation. scELMo has a lighter structure and lower requirements for resources, suggesting a more promising path.

•scELMo leverages large language models to integrate knowledge•scELMo supports multiple settings•scELMo achieves superior performance with fewer resources•scELMo establishes a scalable and interpretable framework

scELMo leverages large language models to integrate knowledge

scELMo supports multiple settings

scELMo achieves superior performance with fewer resources

scELMo establishes a scalable and interpretable framework

Single-cell technologies allow scientists to measure the activity of thousands of genes in individual cells, revealing how tissues develop, age, and respond to disease. Yet, analyzing these massive datasets often demands substantial computing resources and specialized expertise. Our method, single-cell embedding from language models (scELMo), offers an accessible solution by harnessing large language models—the same artificial intelligence systems behind modern chatbots—to interpret biological information. Instead of training large models from scratch, scELMo uses pre-trained language models’ knowledge of gene functions and biological concepts to generate detailed numerical representations, or embeddings, of genes. These embeddings capture complex biological information and can be integrated with cellular data to facilitate tasks such as identifying cell types, understanding developmental processes, or exploring disease mechanisms. By uniting advances in computational linguistics and genomics, scELMo transforms language models into engines of biological discovery, expanding access to powerful single-cell analysis tools and accelerating the pace of biomedical insight.

scELMo introduces a way to analyze massive single-cell datasets by harnessing large language models to summarize biological knowledge about each gene. These summaries are transformed into mathematical embeddings that integrate with cell data, enabling efficient cell grouping, batch correction, and treatment prediction. By merging language understanding with biological data, scELMo reduces computational demands and democratizes advanced analysis—offering a faster, more accessible path to discoveries that could inform new therapies and deepen understanding of human health.