MASC: A Tool for Mutation-Based Evaluation of Static Crypto-API Misuse Detectors

TL;DR

MASC is a versatile tool that uses mutation analysis to evaluate the effectiveness of static crypto-API misuse detectors, revealing undocumented flaws and aiding developers in improving security tools.

Contribution

This paper introduces MASC, a novel mutation-based evaluation framework with customizable operators and scopes for assessing crypto-API misuse detectors.

Findings

Discovered 19 undocumented flaws in existing crypto-detectors.

Developed 12 mutation operators and 3 mutation scopes for comprehensive testing.

Provided a user-friendly tool with CLI and web interfaces.

Abstract

While software engineers are optimistically adopting crypto-API misuse detectors (or crypto-detectors) in their software development cycles, this momentum must be accompanied by a rigorous understanding of crypto-detectors' effectiveness at finding crypto-API misuses in practice. This demo paper presents the technical details and usage scenarios of our tool, namely Mutation Analysis for evaluating Static Crypto-API misuse detectors (MASC). We developed $12$ generalizable, usage based mutation operators and three mutation scopes, namely Main Scope, Similarity Scope, and Exhaustive Scope, which can be used to expressively instantiate compilable variants of the crypto-API misuse cases. Using MASC, we evaluated nine major crypto-detectors, and discovered $19$ unique, undocumented flaws. We designed MASC to be configurable and user-friendly; a user can configure the parameters to change the nature of generated mutations. Furthermore, MASC comes with both Command Line Interface and Web-based front-end, making it practical for users of different levels of expertise.