Generating and Visualizing Trace Link Explanations

TL;DR

This paper introduces a novel NLP pipeline that generates and visualizes explanations for trace links in software engineering, helping non-experts understand link semantics and evaluate their correctness.

Contribution

It presents a new NLP-based method for generating and visualizing trace link explanations, enhancing interpretability for non-expert users in diverse domains.

Findings

Explanations improved non-experts' understanding of trace links.

The approach achieved high coverage and correctness in three domain applications.

User study showed increased ability to evaluate link correctness with explanations.

Abstract

Recent breakthroughs in deep-learning (DL) approaches have resulted in the dynamic generation of trace links that are far more accurate than was previously possible. However, DL-generated links lack clear explanations, and therefore non-experts in the domain can find it difficult to understand the underlying semantics of the link, making it hard for them to evaluate the link's correctness or suitability for a specific software engineering task. In this paper we present a novel NLP pipeline for generating and visualizing trace link explanations. Our approach identifies domain-specific concepts, retrieves a corpus of concept-related sentences, mines concept definitions and usage examples, and identifies relations between cross-artifact concepts in order to explain the links. It applies a post-processing step to prioritize the most likely acronyms and definitions and to eliminate non-relevant ones. We evaluate our approach using project artifacts from three different domains of interstellar telescopes, positive train control, and electronic health-care systems, and then report coverage, correctness, and potential utility of the generated definitions. We design and utilize an explanation interface which leverages concept definitions and relations to visualize and explain trace link rationales, and we report results from a user study that was conducted to evaluate the effectiveness of the explanation interface. Results show that the explanations presented in the interface helped non-experts to understand the underlying semantics of a trace link and improved their ability to vet the correctness of the link.