But that's not why: Inference adjustment by interactive prototype revision

TL;DR

This paper introduces a method for users to interactively revise and correct the reasoning process of prototype-based models, improving interpretability and removing reliance on unreasonable features without sacrificing accuracy.

Contribution

It proposes interactive schemes for prototype adjustment in deep models, enabling non-expert users to correct model reasoning by removing faulty prototypes.

Findings

Prototypical-part models facilitate semantic interpretation of prototypes.

Interactive rejection of prototypes improves model reasoning.

Model accuracy remains stable after prototype adjustments.

Abstract

Despite significant advances in machine learning, decision-making of artificial agents is still not perfect and often requires post-hoc human interventions. If the prediction of a model relies on unreasonable factors it is desirable to remove their effect. Deep interactive prototype adjustment enables the user to give hints and correct the model's reasoning. In this paper, we demonstrate that prototypical-part models are well suited for this task as their prediction is based on prototypical image patches that can be interpreted semantically by the user. It shows that even correct classifications can rely on unreasonable prototypes that result from confounding variables in a dataset. Hence, we propose simple yet effective interaction schemes for inference adjustment: The user is consulted interactively to identify faulty prototypes. Non-object prototypes can be removed by prototype masking or a custom mode of deselection training. Interactive prototype rejection allows machine learning na\"{i}ve users to adjust the logic of reasoning without compromising the accuracy.