What can AI do for me: Evaluating Machine Learning Interpretations in Cooperative Play

TL;DR

This paper evaluates the effectiveness of machine learning interpretations in a real human-AI cooperative setting using a Quizbowl question answering task, focusing on improving human performance and providing design insights for NLP human-in-the-loop systems.

Contribution

It introduces a real-world evaluation framework for ML interpretability in human-AI collaboration, specifically in natural language processing tasks, with insights from experiments involving experts and novices.

Findings

Interpretations can significantly improve human performance in cooperative tasks

Different interpretation methods vary in effectiveness for experts and novices

Provides design guidance for NLP human-in-the-loop applications

Abstract

Machine learning is an important tool for decision making, but its ethical and responsible application requires rigorous vetting of its interpretability and utility: an understudied problem, particularly for natural language processing models. We propose an evaluation of interpretation on a real task with real human users, where the effectiveness of interpretation is measured by how much it improves human performance. We design a grounded, realistic human-computer cooperative setting using a question answering task, Quizbowl. We recruit both trivia experts and novices to play this game with computer as their teammate, who communicates its prediction via three different interpretations. We also provide design guidance for natural language processing human-in-the-loop settings.