SymbolicData:SDEval - Benchmarking for Everyone

TL;DR

SDeval is a flexible benchmarking toolkit for computer algebra systems, enabling reproducible and extendable benchmarks, and addressing unique challenges in the field.

Contribution

It introduces SDeval, a comprehensive tool for benchmarking in computer algebra, facilitating reproducibility and extensibility in research evaluations.

Findings

SDeval simplifies benchmarking processes in computer algebra.

It enables reproducibility of published benchmarks.

The toolkit is adaptable to various communities.

Abstract

In this paper we will present SDeval, a software project that contains tools for creating and running benchmarks with a focus on problems in computer algebra. It is built on top of the Symbolic Data project, able to translate problems in the database into executable code for various computer algebra systems. The included tools are designed to be very flexible to use and to extend, such that they can be utilized even in contexts of other communities. With the presentation of SDEval, we will also address particularities of benchmarking in the field of computer algebra. Furthermore, with SDEval, we provide a feasible and automatizable way of reproducing benchmarks published in current research works, which appears to be a difficult task in general due to the customizability of the available programs. We will simultaneously present the current developments in the Symbolic Data project.