Explanation-Based Human Debugging of NLP Models: A Survey

TL;DR

This survey reviews explanation-based human debugging (EBHD) techniques for NLP models, focusing on how explanations facilitate human feedback to identify and fix model bugs, and discusses future research challenges.

Contribution

It categorizes existing EBHD work in NLP along three dimensions and highlights open problems and future directions in the field.

Findings

EBHD components influence user feedback quality

Different EBHD workflows impact debugging effectiveness

Open problems include improving explanation clarity and user engagement

Abstract

Debugging a machine learning model is hard since the bug usually involves the training data and the learning process. This becomes even harder for an opaque deep learning model if we have no clue about how the model actually works. In this survey, we review papers that exploit explanations to enable humans to give feedback and debug NLP models. We call this problem explanation-based human debugging (EBHD). In particular, we categorize and discuss existing work along three dimensions of EBHD (the bug context, the workflow, and the experimental setting), compile findings on how EBHD components affect the feedback providers, and highlight open problems that could be future research directions.