An MLCommons Scientific Benchmarks Ontology
Ben Hawks, Gregor von Laszewski, Matthew D. Sinclair, Marco Colombo, Shivaram Venkataraman, Rutwik Jain, Yiwei Jiang, Nhan Tran, Geoffrey Fox

TL;DR
This paper presents a comprehensive ontology for scientific machine learning benchmarks, unifying diverse efforts into a standardized, extensible framework to facilitate reproducible, cross-domain benchmarking across multiple scientific disciplines.
Contribution
It introduces a unified, community-driven ontology that consolidates various scientific benchmarks into a scalable, extensible taxonomy with an open submission and evaluation process.
Findings
Consolidates diverse benchmarks into a single taxonomy.
Provides an open, community-driven submission and evaluation process.
Supports future scientific and AI/ML motifs.
Abstract
Scientific machine learning research spans diverse domains and data modalities, yet existing benchmark efforts remain siloed and lack standardization. This makes novel and transformative applications of machine learning to critical scientific use-cases more fragmented and less clear in pathways to impact. This paper introduces an ontology for scientific benchmarking developed through a unified, community-driven effort that extends the MLCommons ecosystem to cover physics, chemistry, materials science, biology, climate science, and more. Building on prior initiatives such as XAI-BENCH, FastML Science Benchmarks, PDEBench, and the SciMLBench framework, our effort consolidates a large set of disparate benchmarks and frameworks into a single taxonomy of scientific, application, and system-level benchmarks. New benchmarks can be added through an open submission workflow coordinated by the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsScientific Computing and Data Management · Machine Learning in Materials Science · Research Data Management Practices
