ProtoQuant: Quantization of Prototypical Parts For General and Fine-Grained Image Classification

TL;DR

ProtoQuant introduces a quantization-based approach to stabilize prototypes in prototypical parts models, enabling scalable, interpretable, and accurate image classification on large datasets like ImageNet.

Contribution

It proposes a novel latent vector quantization method that maintains prototype stability without backbone finetuning, improving scalability and interpretability.

Findings

Achieves competitive accuracy on ImageNet and fine-grained datasets.

Maintains stable, grounded prototypes resistant to perturbations.

Scales efficiently to large datasets without backbone updates.

Abstract

Prototypical parts-based models offer a "this looks like that" paradigm for intrinsic interpretability, yet they typically struggle with ImageNet-scale generalization and often require computationally expensive backbone finetuning. Furthermore, existing methods frequently suffer from "prototype drift," where learned prototypes lack tangible grounding in the training distribution and change their activation under small perturbations. We present ProtoQuant, a novel architecture that achieves prototype stability and grounded interpretability through latent vector quantization. By constraining prototypes to a discrete learned codebook within the latent space, we ensure they remain faithful representations of the training data without the need to update the backbone. This design allows ProtoQuant to function as an efficient, interpretable head that scales to large-scale datasets. We evaluate ProtoQuant on ImageNet and several fine-grained benchmarks (CUB-200, Cars-196). Our results demonstrate that ProtoQuant achieves competitive classification accuracy while generalizing to ImageNet and comparable interpretability metrics to other prototypical-parts-based methods.