How Does Users' App Knowledge Influence the Preferred Level of Detail and Format of Software Explanations?

TL;DR

This study explores how users' app knowledge and demographics influence their preferred explanation formats and detail levels in software, highlighting the importance of adaptive, user-centered explanations.

Contribution

It reveals that explanation preferences are weakly affected by app-specific knowledge but significantly shaped by demographic factors, informing adaptive explanation system design.

Findings

Users prefer moderately detailed explanations in short text format.

No direct link between app knowledge and explanation detail preference.

Demographic factors influence explanation preferences and application confidence.

Abstract

Context and Motivation: Due to their increasing complexity, everyday software systems are becoming increasingly opaque for users. A frequently adopted method to address this difficulty is explainability, which aims to make systems more understandable and usable. Question/problem: However, explanations can also lead to unnecessary cognitive load. Therefore, adapting explanations to the actual needs of a user is a frequently faced challenge. Principal ideas/results: This study investigates factors influencing users' preferred the level of detail and the form of an explanation (e.g., short text or video tutorial) in software. We conducted an online survey with 58 participants to explore relationships between demographics, software usage, app-specific knowledge, as well as their preferred explanation form and level of detail. The results indicate that users prefer moderately detailed explanations in short text formats. Correlation analyses revealed no relationship between app-specific knowledge and the preferred level of detail of an explanation, but an influence of demographic aspects (like gender) on app-specific knowledge and its impact on application confidence were observed, pointing to a possible mediated relationship between knowledge and preferences for explanations. Contribution: Our results show that explanation preferences are weakly influenced by app-specific knowledge but shaped by demographic and psychological factors, supporting the development of adaptive explanation systems tailored to user expertise. These findings support requirements analysis processes by highlighting important factors that should be considered in user-centered methods such as personas.