PointEval: On the Impact of Pointer Analysis Frameworks

TL;DR

This paper evaluates and compares two major pointer analysis frameworks, WALA and Doop, on benchmarks, revealing that Doop generally offers better precision and scalability, with Soot's IR often outperforming Wala's IR.

Contribution

The study provides a comprehensive comparison of WALA and Doop frameworks, including their IRs, and introduces PointerBench for validating pointer analysis results.

Findings

Doop outperforms WALA in precision and scalability.

Soot's IR generally yields more precise and scalable analysis than Wala's IR.

PointerBench effectively validates points-to statistics.

Abstract

Pointer analysis is a foundational analysis leveraged by various static analyses. Therefore, it gathered wide attention in research for decades. Some pointer analysis frameworks are based on succinct declarative specifications. However, these tools are heterogeneous in terms of the underlying intermediate representation (IR), heap abstraction, and programming methodology. This situation complicates a fair comparison of these frameworks and thus hinders further research. Consequently, the literature lacks an evaluation of the strengths and weaknesses of these tools. In this work, we evaluate two major frameworks for pointer analysis, WALA and Doop, on the DaCapo set of benchmarks. We compare the pointer analyses available in Wala and Doop, and conclude that---even though based on a declarative specification---Doop provides a better pointer analysis than Wala in terms of precision and scalability. We also compare the two IRs used in Doop, i.e., Jimple from the Soot framework and IR from the Wala framework. Our evaluation shows that in the majority of the benchmarks Soot's IR gives a more precise and scalable pointer analysis. Finally, we propose a micro-benchmark \emph{PointerBench}, for which we manually validate the points-to statistics to evaluate the results of these tools.