Explainability via Interactivity? Supporting Nonexperts' Sensemaking of Pretrained CNN by Interacting with Their Daily Surroundings

TL;DR

This paper introduces a mobile app that helps nonexperts understand pretrained CNNs by interacting with their environment and visualizing model decisions, enhancing their AI comprehension.

Contribution

It presents a novel interactive tool using Class Activation Maps to support nonexpert users in making sense of CNNs through real-world interactions.

Findings

Supports nonexperts in understanding CNN capabilities and limitations

Enhances sensemaking through interactive visualizations

Facilitates playful learning in educational settings

Abstract

Current research on Explainable AI (XAI) heavily targets on expert users (data scientists or AI developers). However, increasing importance has been argued for making AI more understandable to nonexperts, who are expected to leverage AI techniques, but have limited knowledge about AI. We present a mobile application to support nonexperts to interactively make sense of Convolutional Neural Networks (CNN); it allows users to play with a pretrained CNN by taking pictures of their surrounding objects. We use an up-to-date XAI technique (Class Activation Map) to intuitively visualize the model's decision (the most important image regions that lead to a certain result). Deployed in a university course, this playful learning tool was found to support design students to gain vivid understandings about the capabilities and limitations of pretrained CNNs in real-world environments. Concrete examples of students' playful explorations are reported to characterize their sensemaking processes reflecting different depths of thought.