SlimSeg: Slimmable Semantic Segmentation with Boundary Supervision

TL;DR

SlimSeg introduces a flexible, slimmable semantic segmentation framework that adapts inference capacity dynamically, utilizing boundary supervision and knowledge distillation to enhance performance across different model sizes.

Contribution

The paper presents a novel slimmable segmentation method with boundary supervision, enabling flexible accuracy-efficiency tradeoffs and improved performance over independent models.

Findings

Flexible models outperform independent counterparts.

Boundary supervision enhances segmentation near semantic borders.

Effective across multiple mainstream networks and benchmarks.

Abstract

Accurate semantic segmentation models typically require significant computational resources, inhibiting their use in practical applications. Recent works rely on well-crafted lightweight models to achieve fast inference. However, these models cannot flexibly adapt to varying accuracy and efficiency requirements. In this paper, we propose a simple but effective slimmable semantic segmentation (SlimSeg) method, which can be executed at different capacities during inference depending on the desired accuracy-efficiency tradeoff. More specifically, we employ parametrized channel slimming by stepwise downward knowledge distillation during training. Motivated by the observation that the differences between segmentation results of each submodel are mainly near the semantic borders, we introduce an additional boundary guided semantic segmentation loss to further improve the performance of each submodel. We show that our proposed SlimSeg with various mainstream networks can produce flexible models that provide dynamic adjustment of computational cost and better performance than independent models. Extensive experiments on semantic segmentation benchmarks, Cityscapes and CamVid, demonstrate the generalization ability of our framework.