ProtoSeg: Interpretable Semantic Segmentation with Prototypical Parts

TL;DR

ProtoSeg is an interpretable semantic segmentation model that uses training set patches for predictions, discovering semantic concepts and maintaining high accuracy, with demonstrated effectiveness on Pascal VOC and Cityscapes datasets.

Contribution

It introduces ProtoSeg, a novel approach combining prototypical parts with diversity loss to enhance interpretability and accuracy in semantic segmentation.

Findings

ProtoSeg achieves accuracy comparable to baseline models.

It discovers semantic concepts beyond standard models.

Experimental results confirm its precision and transparency.

Abstract

We introduce ProtoSeg, a novel model for interpretable semantic image segmentation, which constructs its predictions using similar patches from the training set. To achieve accuracy comparable to baseline methods, we adapt the mechanism of prototypical parts and introduce a diversity loss function that increases the variety of prototypes within each class. We show that ProtoSeg discovers semantic concepts, in contrast to standard segmentation models. Experiments conducted on Pascal VOC and Cityscapes datasets confirm the precision and transparency of the presented method.