Visibility of Domain Elements in the Elicitation Process: A Family of Empirical Studies

TL;DR

This study investigates factors influencing analysts' ability to identify domain elements during elicitation, revealing that visibility depends on elicitation methods and domain complexity, rather than analyst experience or interaction quality.

Contribution

The paper presents empirical evidence on how elicitation techniques and domain attributes affect element visibility, highlighting areas for improving elicitation documentation.

Findings

Visibility increases when analysts discuss knowledge gained during elicitation.

Domain complexity and element importance influence visibility.

Experience and client interaction do not significantly affect visibility.

Abstract

Background: Various factors determine analyst effectiveness during elicitation. While the literature suggests that elicitation technique and time are influential factors, other attributes could also play a role. Aim: Determine aspects that may have an influence on analysts' ability to identify certain elements of the problem domain. Methodology: We conducted 14 quasi-experiments, inquiring 134 subjects about two problem domains. For each problem domain, we calculated whether the experimental subjects identified the problem domain elements (concepts, processes, and requirements), i.e., the degree to which these domain elements were visible. Results: Domain element visibility does not appear to be related to either analyst experience or analyst-client interaction. Domain element visibility depends on how analysts provide the elicited information: when asked about the knowledge acquired during elicitation, domain element visibility dramatically increases compared to the information they provide using a written report. Conclusions: Further research is required to replicate our results. However, the finding that analysts have difficulty reporting the information they have acquired is useful for identifying alternatives for improving the documentation of elicitation results. We found evidence that other issues, like domain complexity, the relative importance of different elements within the domain, and the interview script, also seem influential.