Soliciting Human-in-the-Loop User Feedback for Interactive Machine Learning Reduces User Trust and Impressions of Model Accuracy

TL;DR

This study explores how allowing users to give feedback to interactive AI systems can decrease their trust and perceived accuracy, even if the system actually improves.

Contribution

It provides empirical evidence that human-in-the-loop feedback can negatively impact user trust and perceptions, highlighting a critical consideration for system design.

Findings

User trust decreased after providing feedback.

Perception of system accuracy declined regardless of actual performance.

Feedback influenced user impressions more than system improvements.

Abstract

Mixed-initiative systems allow users to interactively provide feedback to potentially improve system performance. Human feedback can correct model errors and update model parameters to dynamically adapt to changing data. Additionally, many users desire the ability to have a greater level of control and fix perceived flaws in systems they rely on. However, how the ability to provide feedback to autonomous systems influences user trust is a largely unexplored area of research. Our research investigates how the act of providing feedback can affect user understanding of an intelligent system and its accuracy. We present a controlled experiment using a simulated object detection system with image data to study the effects of interactive feedback collection on user impressions. The results show that providing human-in-the-loop feedback lowered both participants' trust in the system and their perception of system accuracy, regardless of whether the system accuracy improved in response to their feedback. These results highlight the importance of considering the effects of allowing end-user feedback on user trust when designing intelligent systems.