Teaching Humans When To Defer to a Classifier via Exemplars

TL;DR

This paper introduces an exemplar-based teaching method to help humans develop accurate mental models of AI decision-making, improving collaboration and task performance in complex tasks.

Contribution

It proposes a novel parameterization of human mental models and derives a near-optimal teaching strategy for guiding humans on when to rely on AI.

Findings

Improved human-AI collaboration in multi-hop question answering.

Enhanced human understanding of AI strengths and weaknesses.

Validated effectiveness through crowd worker experiments.

Abstract

Expert decision makers are starting to rely on data-driven automated agents to assist them with various tasks. For this collaboration to perform properly, the human decision maker must have a mental model of when and when not to rely on the agent. In this work, we aim to ensure that human decision makers learn a valid mental model of the agent's strengths and weaknesses. To accomplish this goal, we propose an exemplar-based teaching strategy where humans solve the task with the help of the agent and try to formulate a set of guidelines of when and when not to defer. We present a novel parameterization of the human's mental model of the AI that applies a nearest neighbor rule in local regions surrounding the teaching examples. Using this model, we derive a near-optimal strategy for selecting a representative teaching set. We validate the benefits of our teaching strategy on a multi-hop question answering task using crowd workers and find that when workers draw the right lessons from the teaching stage, their task performance improves, we furthermore validate our method on a set of synthetic experiments.