A Broad Comparative Evaluation of x86-64 Binary Rewriters

TL;DR

This paper provides a comprehensive evaluation of ten popular x86-64 binary rewriters, assessing their generality, reliability, performance, and success predictors to establish a shared understanding of their capabilities.

Contribution

It offers the first broad comparative analysis of binary rewriters, including performance, reliability, and success prediction, facilitating informed tool selection and future research.

Findings

Rewriters vary significantly in generality and reliability.

A simple decision tree can predict rewriting success with high accuracy.

The study provides open-source tools and data for further research.

Abstract

Binary rewriting is a rapidly-maturing technique for modifying software for instrumentation, customization, optimization, and hardening without access to source code. Unfortunately, the practical applications of binary rewriting tools are often unclear to users because their limitations are glossed over in the literature. This, among other challenges, has prohibited the widespread adoption of these tools. To address this shortcoming, we collect ten popular binary rewriters and assess their generality across a broad range of input binary classes and the functional reliability of the resulting rewritten binaries. Additionally, we evaluate the performance of the rewriters themselves as well as the rewritten binaries they produce. The goal of this broad evaluation is to establish a shared context for future research and development of binary rewriting tools by providing a state of the practice for their capabilities. To support potential binary rewriter users, we also identify input binary features that are predictive of tool success and show that a simple decision tree model can accurately predict whether a particular tool can rewrite a target binary. The binary rewriters, our corpus of 3344 sample binaries, and the evaluation infrastructure itself are all freely available as open-source software.