$\mu$SE: Mutation-based Evaluation of Security-focused Static Analysis Tools for Android

TL;DR

$SE is a mutation-based evaluation tool for Android static analysis security tools, systematically uncovering flaws and enabling customization to improve robustness assessment.

Contribution

It introduces a mutation testing approach tailored for evaluating Android security static analysis tools, revealing undocumented flaws and supporting extensibility.

Findings

Found 25 previously undocumented flaws in static data leak detection tools.

Developed four mutation schemes for targeted analysis.

Enabled customization and filtering to improve evaluation efficiency.

Abstract

This demo paper presents the technical details and usage scenarios of $\mu$SE: a mutation-based tool for evaluating security-focused static analysis tools for Android. Mutation testing is generally used by software practitioners to assess the robustness of a given test-suite. However, we leverage this technique to systematically evaluate static analysis tools and uncover and document soundness issues. $\mu$SE's analysis has found 25 previously undocumented flaws in static data leak detection tools for Android. $\mu$SE offers four mutation schemes, namely Reachability, Complex-reachability, TaintSink, and ScopeSink, which determine the locations of seeded mutants. Furthermore, the user can extend $\mu$SE by customizing the API calls targeted by the mutation analysis. $\mu$SE is also practical, as it makes use of filtering techniques based on compilation and execution criteria that reduces the number of ineffective mutations.