COMET: Benchmark for Comprehensive Biological Multi-omics Evaluation Tasks and Language Models

TL;DR

COMET is the first comprehensive benchmark designed to evaluate machine learning models across diverse biological multi-omics tasks, facilitating better model selection and advancing integrated biological data analysis.

Contribution

It introduces a new multi-omics benchmark with curated datasets and evaluates existing language models, providing insights into their effectiveness for complex biological data integration.

Findings

Existing models show varied performance across omics tasks.

Multi-omics models outperform single-omics models in integration tasks.

Benchmark highlights key challenges and future directions in multi-omics research.

Abstract

As key elements within the central dogma, DNA, RNA, and proteins play crucial roles in maintaining life by guaranteeing accurate genetic expression and implementation. Although research on these molecules has profoundly impacted fields like medicine, agriculture, and industry, the diversity of machine learning approaches-from traditional statistical methods to deep learning models and large language models-poses challenges for researchers in choosing the most suitable models for specific tasks, especially for cross-omics and multi-omics tasks due to the lack of comprehensive benchmarks. To address this, we introduce the first comprehensive multi-omics benchmark COMET (Benchmark for Biological COmprehensive Multi-omics Evaluation Tasks and Language Models), designed to evaluate models across single-omics, cross-omics, and multi-omics tasks. First, we curate and develop a diverse collection of downstream tasks and datasets covering key structural and functional aspects in DNA, RNA, and proteins, including tasks that span multiple omics levels. Then, we evaluate existing foundational language models for DNA, RNA, and proteins, as well as the newly proposed multi-omics method, offering valuable insights into their performance in integrating and analyzing data from different biological modalities. This benchmark aims to define critical issues in multi-omics research and guide future directions, ultimately promoting advancements in understanding biological processes through integrated and different omics data analysis.