Second-Guessing in Tracing Tasks Considered Harmful?

TL;DR

This study investigates whether modifying tracing software to prevent certain analyst behaviors can improve accuracy in software trace recovery tasks, addressing limitations of automated methods and human errors.

Contribution

It introduces a three-step experimental approach and reports preliminary findings on software modifications aimed at reducing analyst errors in trace recovery.

Findings

Modified RETRO.NET to curtail specific behaviors

Preliminary results suggest behavioral modifications may reduce errors

Study advances understanding of human factors in trace analysis

Abstract

[Context and motivation] Trace matrices are lynch pins for the development of mission- and safety-critical software systems and are useful for all software systems, yet automated methods for recovering trace links are far from perfect. This limitation makes the job of human analysts who must vet recovered trace links more difficult. [Question/Problem] Earlier studies suggested that certain analyst behaviors when performing trace recovery tasks lead to decreased accuracy of recovered trace relationships. We propose a three-step experimental study to: (a) determine if there really are behaviors that lead to errors of judgment for analysts, (b) enhance the requirements tracing software to curtail such behaviors, and (c) determine if curtailing such behaviors results in increased accuracy. [Principal ideas/results] We report on a preliminary study we undertook in which we modified the user interface of RETRO.NET to curtail two behaviors indicated by the earlier work. We report on observed results. [Contributions] We describe and discuss a major study of potentially unwanted analyst behaviors and present results of a preliminary study toward determining if curbing these behaviors with enhancements to tracing software leads to fewer human errors.