An Empirical Investigation on the Challenges of Creating Custom Static Analysis Rules for Defect Localization

TL;DR

This study empirically investigates the challenges faced by novice maintainers when applying Pattern-Driven Maintenance (PDM) to create custom static analysis rules for defect localization, highlighting the importance of training and abstraction levels.

Contribution

It provides empirical insights into the skills, challenges, and factors influencing the effective application of PDM for static analysis rule creation by maintainers.

Findings

Prior knowledge impacts rule creation efficiency.

Training and abstraction levels ease rule programming.

Proper selection of maintainers improves PDM effectiveness.

Abstract

Background: Custom static analysis rules, i.e., rules specific for one or more applications, have been successfully applied to perform corrective and preventive software maintenance. Pattern-Driven Maintenance (PDM) is a method designed to support the creation of such rules during software maintenance. However, as PDM was recently proposed, few maintainers have reported on its usage. Hence, the challenges and skills needed to apply PDM properly are unknown. Aims: In this paper, we investigate the challenges faced by maintainers on applying PDM for creating custom static analysis rules for defect localization. Method: We conducted an observational study on novice maintainers creating custom static analysis rules by applying PDM. The study was divided into three tasks: (i) identifying a defect pattern, (ii) programming a static analysis rule to locate instances of the pattern, and (iii) verifying the located instances. We analyzed the efficiency and acceptance of maintainers on applying PDM and their comments on task challenges. Results: We observed that previous knowledge on debugging, the subject software, and related technologies influenced the performance of maintainers as well as the time to learn the technology involved in rule programming. Conclusions: The results strengthen our confidence that PDM can help maintainers in producing custom static analysis rules for locating defects. However, a proper selection and training of maintainers is needed to apply PDM effectively. Also, using a higher level of abstraction can ease static analysis rule programming for novice maintainers.