Refining Semantic Segmentation with Superpixel by Transparent Initialization and Sparse Encoder

TL;DR

This paper introduces a novel method combining trainable superpixels with transparent initialization and a sparse encoder to improve semantic segmentation, especially around object edges, with reduced computational costs.

Contribution

It proposes a joint learning framework for semantic segmentation with superpixels, featuring transparent initialization and a sparse encoder to enhance edge accuracy and efficiency.

Findings

TI outperforms other initialization methods

Method improves segmentation edge accuracy

Validated on PASCAL VOC 2012, ADE20K, PASCAL Context

Abstract

Although deep learning greatly improves the performance of semantic segmentation, its success mainly lies in object central areas without accurate edges. As superpixels are a popular and effective auxiliary to preserve object edges, in this paper, we jointly learn semantic segmentation with trainable superpixels. We achieve it with fully-connected layers with Transparent Initialization (TI) and efficient logit consistency using a sparse encoder. The proposed TI preserves the effects of learned parameters of pretrained networks. This avoids a significant increase of the loss of pretrained networks, which otherwise may be caused by inappropriate parameter initialization of the additional layers. Meanwhile, consistent pixel labels in each superpixel are guaranteed by logit consistency. The sparse encoder with sparse matrix operations substantially reduces both the memory requirement and the computational complexity. We demonstrated the superiority of TI over other parameter initialization methods and tested its numerical stability. The effectiveness of our proposal was validated on PASCAL VOC 2012, ADE20K, and PASCAL Context showing enhanced semantic segmentation edges. With quantitative evaluations on segmentation edges using performance ratio and F-measure, our method outperforms the state-of-the-art.