Revealing Performance Issues in Server-side WebAssembly Runtimes via Differential Testing

TL;DR

This paper introduces WarpDiff, a differential testing approach to identify and analyze performance issues in server-side WebAssembly runtimes, revealing seven confirmed problems and aiding future improvements.

Contribution

It presents a novel differential testing method for detecting performance issues in server-side WebAssembly runtimes, validated on five popular runtimes with real abnormal cases.

Findings

Identified 10 abnormal performance cases in five WebAssembly runtimes.

Confirmed seven performance issues through developer validation.

Demonstrated WarpDiff's effectiveness in revealing runtime performance problems.

Abstract

WebAssembly (Wasm) is a bytecode format originally serving as a compilation target for Web applications. It has recently been used increasingly on the server side, e.g., providing a safer, faster, and more portable alternative to Linux containers. With the popularity of server-side Wasm applications, it is essential to study performance issues (i.e., abnormal latency) in Wasm runtimes, as they may cause a significant impact on server-side applications. However, there is still a lack of attention to performance issues in server-side Wasm runtimes. In this paper, we design a novel differential testing approach WarpDiff to identify performance issues in server-side Wasm runtimes. The key insight is that in normal cases, the execution time of the same test case on different Wasm runtimes should follow an oracle ratio. We identify abnormal cases where the execution time ratio significantly deviates from the oracle ratio and subsequently locate the Wasm runtimes that cause the performance issues. We apply WarpDiff to test five popular server-side Wasm runtimes using 123 test cases from the LLVM test suite and demonstrate the top 10 abnormal cases we identified. We further conduct an in-depth analysis of these abnormal cases and summarize seven performance issues, all of which have been confirmed by the developers. We hope our work can inspire future investigation on improving Wasm runtime implementation and thus promoting the development of server-side Wasm applications.