Fault Localization for Declarative Models in Alloy

TL;DR

This paper presents AlloyFL, a novel fault localization framework for Alloy declarative models, utilizing multiple test formulas and AST-level granularity, with evaluation showing mutation-based techniques outperform others.

Contribution

Introduces AlloyFL, the first fault localization techniques for Alloy models that leverage multiple test formulas and operate at AST node granularity.

Findings

Mutation-based techniques are more accurate than spectrum-based and unsat core methods.

AlloyFL evaluated on 38 real faulty models and 9000 mutants.

Systematic evaluation demonstrates effectiveness of the proposed techniques.

Abstract

Fault localization is a popular research topic and many techniques have been proposed to locate faults in imperative code, e.g. C and Java. In this paper, we focus on the problem of fault localization for declarative models in Alloy -- a first order relational logic with transitive closure. We introduce AlloyFL, the first set of fault localization techniques for faulty Alloy models which leverages multiple test formulas. AlloyFL is also the first set of fault localization techniques at the AST node granularity. We implements in AlloyFL both spectrum-based and mutation-based fault localization techniques, as well as techniques that are based on Alloy's built-in unsat core. We introduce new metrics to measure the accuracy of AlloyFL and systematically evaluate AlloyFL on 38 real faulty models and 9000 mutant models. The results show that the mutation-based fault localization techniques are significantly more accurate than other types of techniques.