Wasure: A Modular Toolkit for Comprehensive WebAssembly Benchmarking

TL;DR

Wasure is a modular toolkit designed to facilitate comprehensive and systematic benchmarking of WebAssembly performance across diverse platforms, configurations, and applications.

Contribution

It introduces a flexible, extensible toolkit for executing and comparing WebAssembly benchmarks, along with a dynamic analysis of benchmark diversity and coverage.

Findings

Significant variation in code coverage and execution patterns among benchmarks.

Benchmark diversity is crucial for thorough WebAssembly performance evaluation.

Wasure enables more transparent and systematic benchmarking processes.

Abstract

WebAssembly (Wasm) has become a key compilation target for portable and efficient execution across diverse platforms. Benchmarking its performance, however, is a multi-dimensional challenge: it depends not only on the choice of runtime engines, but also on hardware architectures, application domains, source languages, benchmark suites, and runtime configurations. This paper introduces Wasure, a modular and extensible command-line toolkit that simplifies the execution and comparison of WebAssembly benchmarks. To complement performance evaluation, we also conducted a dynamic analysis of the benchmark suites included with Wasure. Our analysis reveals substantial differences in code coverage, control flow, and execution patterns, emphasizing the need for benchmark diversity. Wasure aims to support researchers and developers in conducting more systematic, transparent, and insightful evaluations of WebAssembly engines.