LIMEcraft: Handcrafted superpixel selection and inspection for Visual eXplanations

TL;DR

LIMEcraft is an interactive tool that improves visual explanations of deep learning models by allowing users to select and inspect semantically meaningful image regions, enhancing understanding and safety.

Contribution

It introduces an interactive process for selecting and inspecting superpixels, addressing limitations of static explanations in deep learning interpretability.

Findings

Enhances model safety by inspecting potential biases.

Allows semantic interaction with image explanations.

Improves understanding of model decision processes.

Abstract

The increased interest in deep learning applications, and their hard-to-detect biases result in the need to validate and explain complex models. However, current explanation methods are limited as far as both the explanation of the reasoning process and prediction results are concerned. They usually only show the location in the image that was important for model prediction. The lack of possibility to interact with explanations makes it difficult to verify and understand exactly how the model works. This creates a significant risk when using the model. The risk is compounded by the fact that explanations do not take into account the semantic meaning of the explained objects. To escape from the trap of static and meaningless explanations, we propose a tool and a process called LIMEcraft. LIMEcraft enhances the process of explanation by allowing a user to interactively select semantically consistent areas and thoroughly examine the prediction for the image instance in case of many image features. Experiments on several models show that our tool improves model safety by inspecting model fairness for image pieces that may indicate model bias. The code is available at: http://github.com/MI2DataLab/LIMEcraft