Pixel-Grounded Prototypical Part Networks

TL;DR

This paper introduces PIXPNET, an improved proto-part network that accurately localizes object parts, enhances interpretability, and corrects prior limitations in prototype-based neural networks.

Contribution

It proposes new receptive field constraints and pixel space mapping for ProtoPartNNs, making them truly localize object parts and improving interpretability without losing accuracy.

Findings

PIXPNET outperforms previous models in part localization accuracy.

Enhanced interpretability demonstrated through more meaningful visualizations.

Achieves comparable or better classification accuracy than prior ProtoPartNNs.

Abstract

Prototypical part neural networks (ProtoPartNNs), namely PROTOPNET and its derivatives, are an intrinsically interpretable approach to machine learning. Their prototype learning scheme enables intuitive explanations of the form, this (prototype) looks like that (testing image patch). But, does this actually look like that? In this work, we delve into why object part localization and associated heat maps in past work are misleading. Rather than localizing to object parts, existing ProtoPartNNs localize to the entire image, contrary to generated explanatory visualizations. We argue that detraction from these underlying issues is due to the alluring nature of visualizations and an over-reliance on intuition. To alleviate these issues, we devise new receptive field-based architectural constraints for meaningful localization and a principled pixel space mapping for ProtoPartNNs. To improve interpretability, we propose additional architectural improvements, including a simplified classification head. We also make additional corrections to PROTOPNET and its derivatives, such as the use of a validation set, rather than a test set, to evaluate generalization during training. Our approach, PIXPNET (Pixel-grounded Prototypical part Network), is the only ProtoPartNN that truly learns and localizes to prototypical object parts. We demonstrate that PIXPNET achieves quantifiably improved interpretability without sacrificing accuracy.