Distinguishability-guided Test Program Generation for WebAssembly Runtime Performance Testing

TL;DR

This paper introduces WarpGen, a novel test program generation method for WebAssembly runtimes that uses distinguishability to identify performance issues, leading to the discovery of new performance problems.

Contribution

WarpGen is a new approach that synthesizes test programs based on distinguishability, improving performance testing quality for WebAssembly runtimes.

Findings

WarpGen effectively identifies performance issues in WebAssembly runtimes.

WarpGen discovered seven new performance issues across three runtimes.

The approach outperforms baseline methods in test program quality.

Abstract

WebAssembly (Wasm) is a binary instruction format designed as a portable compilation target, which has been widely used on both the web and server sides in recent years. As high performance is a critical design goal of Wasm, it is essential to conduct performance testing for Wasm runtimes. However, existing research on Wasm runtime performance testing still suffers from insufficient high-quality test programs. To solve this problem, we propose a novel test program generation approach WarpGen. It first extracts code snippets from historical issue-triggering test programs as initial operators, then inserts an operator into a seed program to synthesize a new test program. To verify the quality of generated programs, we propose an indicator called distinguishability, which refers to the ability of a test program to distinguish abnormal performance of specific Wasm runtimes. We apply WarpGen for performance testing on four Wasm runtimes and verify its effectiveness compared with baseline approaches. In particular, WarpGen has identified seven new performance issues in three Wasm runtimes.