Explainability Requires Interactivity

TL;DR

This paper introduces an interactive framework for understanding complex deep neural network decisions in computer vision, highlighting its advantages over traditional static explanation methods in providing more accurate insights.

Contribution

The paper presents a novel interactive approach that enables users to explore and test neural network decisions, improving interpretability over static explanation techniques.

Findings

Interactive approach offers deeper insights into model decisions.

Static explanations can mislead users about model behavior.

Interactivity reduces the risk of misinterpretation in model explanations.

Abstract

When explaining the decisions of deep neural networks, simple stories are tempting but dangerous. Especially in computer vision, the most popular explanation approaches give a false sense of comprehension to its users and provide an overly simplistic picture. We introduce an interactive framework to understand the highly complex decision boundaries of modern vision models. It allows the user to exhaustively inspect, probe, and test a network's decisions. Across a range of case studies, we compare the power of our interactive approach to static explanation methods, showing how these can lead a user astray, with potentially severe consequences.