Comprehensive Attribution: Inherently Explainable Vision Model with Feature Detector

TL;DR

This paper proposes a new inherently explainable vision model that improves attribution accuracy and robustness by addressing key issues like incompleteness and interlocking through a novel training objective and a pre-trained discriminative feature detector.

Contribution

It introduces a new objective and a pre-trained detector to enhance feature coverage and break the interlocking problem in explainable attribution models.

Findings

Achieves higher prediction accuracy than black-box models

Produces attribution maps with high feature coverage and localization

Demonstrates improved fidelity and robustness

Abstract

As deep vision models' popularity rapidly increases, there is a growing emphasis on explanations for model predictions. The inherently explainable attribution method aims to enhance the understanding of model behavior by identifying the important regions in images that significantly contribute to predictions. It is achieved by cooperatively training a selector (generating an attribution map to identify important features) and a predictor (making predictions using the identified features). Despite many advancements, existing methods suffer from the incompleteness problem, where discriminative features are masked out, and the interlocking problem, where the non-optimized selector initially selects noise, causing the predictor to fit on this noise and perpetuate the cycle. To address these problems, we introduce a new objective that discourages the presence of discriminative features in the masked-out regions thus enhancing the comprehensiveness of feature selection. A pre-trained detector is introduced to detect discriminative features in the masked-out region. If the selector selects noise instead of discriminative features, the detector can observe and break the interlocking situation by penalizing the selector. Extensive experiments show that our model makes accurate predictions with higher accuracy than the regular black-box model, and produces attribution maps with high feature coverage, localization ability, fidelity and robustness. Our code will be available at \href{https://github.com/Zood123/COMET}{https://github.com/Zood123/COMET}.