Class Based Thresholding in Early Exit Semantic Segmentation Networks

TL;DR

This paper introduces Class Based Thresholding (CBT), a method that reduces computational costs in early exit semantic segmentation models by assigning class-specific thresholds based on neural collapse, maintaining performance.

Contribution

The paper presents a novel class-based thresholding technique that exploits neural collapse to optimize early exit segmentation models for efficiency.

Findings

Reduces computational cost by 23% on Cityscapes and ADE20K datasets.

Maintains mean intersection over union (mIoU) performance.

Effective in early exit semantic segmentation models.

Abstract

We propose Class Based Thresholding (CBT) to reduce the computational cost of early exit semantic segmentation models while preserving the mean intersection over union (mIoU) performance. A key idea of CBT is to exploit the naturally-occurring neural collapse phenomenon. Specifically, by calculating the mean prediction probabilities of each class in the training set, CBT assigns different masking threshold values to each class, so that the computation can be terminated sooner for pixels belonging to easy-to-predict classes. We show the effectiveness of CBT on Cityscapes and ADE20K datasets. CBT can reduce the computational cost by $23\%$ compared to the previous state-of-the-art early exit models.