An MLCommons Scientific Benchmarks Ontology

TL;DR

This paper presents a comprehensive ontology for scientific machine learning benchmarks, unifying diverse efforts into a standardized, extensible framework to facilitate reproducible, cross-domain benchmarking across multiple scientific disciplines.

Contribution

It introduces a unified, community-driven ontology that consolidates various scientific benchmarks into a scalable, extensible taxonomy with an open submission and evaluation process.

Findings

Consolidates diverse benchmarks into a single taxonomy.

Provides an open, community-driven submission and evaluation process.

Supports future scientific and AI/ML motifs.

Abstract

Scientific machine learning research spans diverse domains and data modalities, yet existing benchmark efforts remain siloed and lack standardization. This makes novel and transformative applications of machine learning to critical scientific use-cases more fragmented and less clear in pathways to impact. This paper introduces an ontology for scientific benchmarking developed through a unified, community-driven effort that extends the MLCommons ecosystem to cover physics, chemistry, materials science, biology, climate science, and more. Building on prior initiatives such as XAI-BENCH, FastML Science Benchmarks, PDEBench, and the SciMLBench framework, our effort consolidates a large set of disparate benchmarks and frameworks into a single taxonomy of scientific, application, and system-level benchmarks. New benchmarks can be added through an open submission workflow coordinated by the MLCommons Science Working Group and evaluated against a six-category rating rubric that promotes and identifies high-quality benchmarks, enabling stakeholders to select benchmarks that meet their specific needs. The architecture is extensible, supporting future scientific and AI/ML motifs, and we discuss methods for identifying emerging computing patterns for unique scientific workloads. The MLCommons Science Benchmarks Ontology provides a standardized, scalable foundation for reproducible, cross-domain benchmarking in scientific machine learning. A companion webpage for this work has also been developed as the effort evolves: https://mlcommons-science.github.io/benchmark/