Memory Wrap: a Data-Efficient and Interpretable Extension to Image Classification Models

TL;DR

Memory Wrap is a versatile extension for image classifiers that enhances data efficiency and interpretability through a content-attention mechanism, enabling better performance with limited data and transparent decision explanations.

Contribution

It introduces Memory Wrap, a novel plug-and-play module that improves data efficiency and interpretability of image classification models using memory and attention mechanisms.

Findings

Outperforms standard classifiers with limited data

Achieves comparable performance with full datasets

Provides interpretable explanations via examples and counterfactuals

Abstract

Due to their black-box and data-hungry nature, deep learning techniques are not yet widely adopted for real-world applications in critical domains, like healthcare and justice. This paper presents Memory Wrap, a plug-and-play extension to any image classification model. Memory Wrap improves both data-efficiency and model interpretability, adopting a content-attention mechanism between the input and some memories of past training samples. We show that Memory Wrap outperforms standard classifiers when it learns from a limited set of data, and it reaches comparable performance when it learns from the full dataset. We discuss how its structure and content-attention mechanisms make predictions interpretable, compared to standard classifiers. To this end, we both show a method to build explanations by examples and counterfactuals, based on the memory content, and how to exploit them to get insights about its decision process. We test our approach on image classification tasks using several architectures on three different datasets, namely CIFAR10, SVHN, and CINIC10.