The Role of Domain Expertise in User Trust and the Impact of First Impressions with Intelligent Systems

TL;DR

This study investigates how domain expertise and the sequence of encountering errors influence user trust in intelligent systems, revealing that early errors can harm trust among experts, while novices tend to over-rely regardless of error timing.

Contribution

The paper provides empirical evidence on how domain knowledge and first impressions affect trust dynamics in users interacting with explainable AI systems.

Findings

Early errors reduce trust among domain experts.

Early correct outputs help knowledgeable users adjust trust.

Novice users tend to over-rely regardless of error timing.

Abstract

Domain-specific intelligent systems are meant to help system users in their decision-making process. Many systems aim to simultaneously support different users with varying levels of domain expertise, but prior domain knowledge can affect user trust and confidence in detecting system errors. While it is also known that user trust can be influenced by first impressions with intelligent systems, our research explores the relationship between ordering bias and domain expertise when encountering errors in intelligent systems. In this paper, we present a controlled user study to explore the role of domain knowledge in establishing trust and susceptibility to the influence of first impressions on user trust. Participants reviewed an explainable image classifier with a constant accuracy and two different orders of observing system errors (observing errors in the beginning of usage vs. in the end). Our findings indicate that encountering errors early-on can cause negative first impressions for domain experts, negatively impacting their trust over the course of interactions. However, encountering correct outputs early helps more knowledgable users to dynamically adjust their trust based on their observations of system performance. In contrast, novice users suffer from over-reliance due to their lack of proper knowledge to detect errors.