Towards a Benchmark Framework for Model Order Reduction in the Mathematical Research Data Initiative (MaRDI)

TL;DR

This paper introduces a benchmark framework and a prototype tool, MORB, for fair comparison of model order reduction algorithms within the MaRDI initiative, leveraging community-curated data from MORWiki.

Contribution

It proposes a generic benchmark framework and develops MORB, a prototype tool, to facilitate fair comparison and validation of model order reduction methods using community-curated data.

Findings

MORB supports comparison of linear time-invariant systems.

The framework enables standardized algorithm evaluation.

Community-curated data enhances benchmarking quality.

Abstract

The race for the most efficient, accurate, and universal algorithm in scientific computing drives innovation. At the same time, this healthy competition is only beneficial if the research output is actually comparable to prior results. Fairly comparing algorithms can be a complex endeavor, as the implementation, configuration, compute environment, and test problems need to be well-defined. Due to the increase in computer-based experiments, new infrastructure for facilitating the exchange and comparison of new algorithms is also needed. To this end, we propose a benchmark framework, as a set of generic specifications for comparing implementations of algorithms using test cases native to a community. Its value lies in its ability to fairly compare and validate existing methods for new applications, as well as compare newly developed methods with existing ones. As a prototype for a more general framework, we have begun building a benchmark tool for the model order reduction (MOR) community. The data basis of the tool is the collection of the Model Order Reduction Wiki (MORWiki). The wiki features three main categories: benchmarks, methods, and software. An editorial board curates submissions and patrols edited entries. Data sets for linear and parametric-linear models are already well represented in the existing collection. Data sets for non-linear or procedural models, for which only evaluation data, or codes / algorithmic descriptions, rather than equations, are available, are being added and extended. Properties and interesting characteristics used for benchmark selection and later assessments are recorded in the model metadata. Our tool, the Model Order Reduction Benchmarker (MORB) is under active development for linear time-invariant systems and solvers.