Evolutionary Grammar-Based Fuzzing

TL;DR

EvoGFuzz is an evolutionary approach that optimizes grammar rule probabilities to generate more effective test inputs, significantly improving defect detection and code coverage in real-world applications.

Contribution

It introduces EvoGFuzz, a novel evolutionary method for tuning grammar probabilities to enhance fuzzing effectiveness over traditional probabilistic approaches.

Findings

Achieved up to 48% higher median line coverage.

Detected 11 unique defects, 5 of which were new.

Outperformed baseline probabilistic grammar fuzzing.

Abstract

A fuzzer provides randomly generated inputs to a targeted software to expose erroneous behavior. To efficiently detect defects, generated inputs should conform to the structure of the input format and thus, grammars can be used to generate syntactically correct inputs. In this context, fuzzing can be guided by probabilities attached to competing rules in the grammar, leading to the idea of probabilistic grammar-based fuzzing. However, the optimal assignment of probabilities to individual grammar rules to effectively expose erroneous behavior for individual systems under test is an open research question. In this paper, we present EvoGFuzz, an evolutionary grammar-based fuzzing approach to optimize the probabilities to generate test inputs that may be more likely to trigger exceptional behavior. The evaluation shows the effectiveness of EvoGFuzz in detecting defects compared to probabilistic grammar-based fuzzing (baseline). Applied to ten real-world applications with common input formats (JSON, JavaScript, or CSS3), the evaluation shows that EvoGFuzz achieved a significantly larger median line coverage for all subjects by up to 48% compared to the baseline. Moreover, EvoGFuzz managed to expose 11 unique defects, from which five have not been detected by the baseline.