An Empirical Study on the Distance Metric in Guiding Directed Grey-box Fuzzing

TL;DR

This study empirically evaluates how different distance metrics influence directed grey-box fuzzing performance, revealing limited impact of metric choice and highlighting the need for better mutation strategies.

Contribution

First empirical analysis of various distance metrics in guiding directed grey-box fuzzing, providing insights into their limited effectiveness and limitations of current mutation strategies.

Findings

Different distance metrics show minimal performance variation.

Distance metrics do not effectively measure vulnerability triggering difficulty.

Current mutation strategies have inherent limitations in generating high-quality testcases.

Abstract

Directed grey-box fuzzing (DGF) aims to discover vulnerabilities in specific code areas efficiently. Distance metric, which is used to measure the quality of seed in DGF, is a crucial factor in affecting the fuzzing performance. Despite distance metrics being widely applied in existing DGF frameworks, it remains opaque about how different distance metrics guide the fuzzing process and affect the fuzzing result in practice. In this paper, we conduct the first empirical study to explore how different distance metrics perform in guiding DGFs. Specifically, we systematically discuss different distance metrics in the aspect of calculation method and granularity. Then, we implement different distance metrics based on AFLGo. On this basis, we conduct comprehensive experiments to evaluate the performance of these distance metrics on the benchmarks widely used in existing DGF-related work. The experimental results demonstrate the following insights. First, the difference among different distance metrics with varying methods of calculation and granularities is not significant. Second, the distance metrics may not be effective in describing the difficulty of triggering the target vulnerability. In addition, by scrutinizing the quality of testcases, our research highlights the inherent limitation of existing mutation strategies in generating high-quality testcases, calling for designing effective mutation strategies for directed fuzzing. We open-source the implementation code and experiment dataset to facilitate future research in DGF.