Designing a Direct Feedback Loop between Humans and Convolutional Neural Networks through Local Explanations

TL;DR

This paper introduces DeepFuse, an interactive system that creates a direct feedback loop between CNN engineers and models, enabling diagnosis and revision of vulnerabilities using local explanations, leading to more accurate and reasonable models.

Contribution

DeepFuse is the first interactive tool that facilitates systematic diagnosis and revision of CNN vulnerabilities through local explanations, improving model accuracy and interpretability.

Findings

Participants created more accurate models with DeepFuse.

DeepFuse effectively guides case-based reasoning in CNN diagnosis.

Engineers found DeepFuse practical for improving current XAI practices.

Abstract

The local explanation provides heatmaps on images to explain how Convolutional Neural Networks (CNNs) derive their output. Due to its visual straightforwardness, the method has been one of the most popular explainable AI (XAI) methods for diagnosing CNNs. Through our formative study (S1), however, we captured ML engineers' ambivalent perspective about the local explanation as a valuable and indispensable envision in building CNNs versus the process that exhausts them due to the heuristic nature of detecting vulnerability. Moreover, steering the CNNs based on the vulnerability learned from the diagnosis seemed highly challenging. To mitigate the gap, we designed DeepFuse, the first interactive design that realizes the direct feedback loop between a user and CNNs in diagnosing and revising CNN's vulnerability using local explanations. DeepFuse helps CNN engineers to systemically search "unreasonable" local explanations and annotate the new boundaries for those identified as unreasonable in a labor-efficient manner. Next, it steers the model based on the given annotation such that the model doesn't introduce similar mistakes. We conducted a two-day study (S2) with 12 experienced CNN engineers. Using DeepFuse, participants made a more accurate and "reasonable" model than the current state-of-the-art. Also, participants found the way DeepFuse guides case-based reasoning can practically improve their current practice. We provide implications for design that explain how future HCI-driven design can move our practice forward to make XAI-driven insights more actionable.