CHALLENGER: Training with Attribution Maps

TL;DR

Challenger introduces a novel training module that uses attribution maps to improve neural network regularization, leading to enhanced performance across various data domains and dataset sizes.

Contribution

The paper presents Challenger, a domain-independent module that leverages attribution maps to manipulate relevant input patterns, improving model diversity and performance.

Findings

Significant performance gains on small and large datasets.

State-of-the-art results in vision, NLP, and time series tasks.

Improved model calibration and classification accuracy.

Abstract

We show that utilizing attribution maps for training neural networks can improve regularization of models and thus increase performance. Regularization is key in deep learning, especially when training complex models on relatively small datasets. In order to understand inner workings of neural networks, attribution methods such as Layer-wise Relevance Propagation (LRP) have been extensively studied, particularly for interpreting the relevance of input features. We introduce Challenger, a module that leverages the explainable power of attribution maps in order to manipulate particularly relevant input patterns. Therefore, exposing and subsequently resolving regions of ambiguity towards separating classes on the ground-truth data manifold, an issue that arises particularly when training models on rather small datasets. Our Challenger module increases model performance through building more diverse filters within the network and can be applied to any input data domain. We demonstrate that our approach results in substantially better classification as well as calibration performance on datasets with only a few samples up to datasets with thousands of samples. In particular, we show that our generic domain-independent approach yields state-of-the-art results in vision, natural language processing and on time series tasks.