SoK: Where to Fuzz? Assessing Target Selection Methods in Directed Fuzzing

TL;DR

This paper provides a comprehensive analysis of target selection methods in directed fuzzing, revealing that simple software metrics outperform complex heuristics and highlighting the potential of language models for target prioritization.

Contribution

It introduces a systematic comparison of target selection methods in fuzzing, modeling them as scoring functions and evaluating their effectiveness on a large crash dataset.

Findings

Simple software metrics outperform other heuristics.

Language models show promise for target selection.

Target selection is an orthogonal dimension to fuzzing performance.

Abstract

A common paradigm for improving fuzzing performance is to focus on selected regions of a program rather than its entirety. While previous work has largely explored how these locations can be reached, their selection, that is, the where, has received little attention so far. A common paradigm for improving fuzzing performance is to focus on selected regions of a program rather than its entirety. While previous work has largely explored how these locations can be reached, their selection, that is, the where, has received little attention so far. In this paper, we fill this gap and present the first comprehensive analysis of target selection methods for fuzzing. To this end, we examine papers from leading security and software engineering conferences, identifying prevalent methods for choosing targets. By modeling these methods as general scoring functions, we are able to compare and measure their efficacy on a corpus of more than 1,600 crashes from the OSS-Fuzz project. Our analysis provides new insights for target selection in practice: First, we find that simple software metrics significantly outperform other methods, including common heuristics used in directed fuzzing, such as recently modified code or locations with sanitizer instrumentation. Next to this, we identify language models as a promising choice for target selection. In summary, our work offers a new perspective on directed fuzzing, emphasizing the role of target selection as an orthogonal dimension to improve performance.