WRTester: Differential Testing of WebAssembly Runtimes via Semantic-aware Binary Generation

TL;DR

WRTester is a novel differential testing framework that generates semantic-rich WebAssembly binaries from real-world cases to effectively uncover hidden bugs across different runtimes.

Contribution

It introduces a new method for generating complex Wasm test cases via disassembly and assembly, improving bug detection in WebAssembly runtimes.

Findings

Uncovered 33 unique bugs in popular Wasm runtimes

Outperformed state-of-the-art techniques in efficiency and effectiveness

Confirmed 25 of the reported bugs

Abstract

Wasm runtime is a fundamental component in the Wasm ecosystem, as it directly impacts whether Wasm applications can be executed as expected. Bugs in Wasm runtime bugs are frequently reported, thus our research community has made a few attempts to design automated testing frameworks for detecting bugs in Wasm runtimes. However, existing testing frameworks are limited by the quality of test cases, i.e., they face challenges of generating both semantic-rich and syntactic-correct Wasm binaries, thus complicated bugs cannot be triggered. In this work, we present WRTester, a novel differential testing framework that can generated complicated Wasm test cases by disassembling and assembling of real-world Wasm binaries, which can trigger hidden inconsistencies among Wasm runtimes. For further pinpointing the root causes of unexpected behaviors, we design a runtime-agnostic root cause location method to accurately locate bugs. Extensive evaluation suggests that WRTester outperforms SOTA techniques in terms of both efficiency and effectiveness. We have uncovered 33 unique bugs in popular Wasm runtimes, among which 25 have been confirmed.