Symetra: Visual Analytics for the Parameter Tuning Process of Symbolic Execution Engines

TL;DR

Symetra is a visual analytics tool that enhances human understanding and tuning of symbolic execution engine parameters, leading to improved branch coverage and tuning efficiency.

Contribution

The paper introduces Symetra, a novel visual analytics system that supports human-in-the-loop parameter tuning for symbolic execution engines, providing insights into parameter impacts.

Findings

Experts used Symetra to interpret parameter impacts effectively.

Human-in-the-loop tuning outperformed automated methods in coverage and efficiency.

Symetra facilitated collective analysis and identification of beneficial configurations.

Abstract

Symbolic execution engines such as KLEE automatically generate test cases to maximize branch coverage, but their numerous parameters make it difficult to understand the parameters' impact, leading the user to rely on suboptimal default configurations. While automated tuners have shown promising results, they provide limited insights into why certain configurations work well, motivating the need for Human-in-the-Loop approaches. In this work, we present a visual analytics system, Symetra, designed to support Human-in-the-Loop parameter tuning of symbolic execution engines. To handle a large number of parameters and their configurations, we provide two complementary overviews of their impact on branch coverage values and patterns. Building on these overviews, our system enables collective analysis, allowing the user to contrast groups of configurations and identify differences that may affect branch coverage. We also report on case studies and a Human-in-the-Loop tuning process, demonstrating that experts not only interpreted parameter impacts and identified complementary configurations, but also improved upon fully automated approaches in both branch coverage and tuning efficiency.