Superpixel-enhanced Pairwise Conditional Random Field for Semantic Segmentation

TL;DR

This paper introduces a superpixel-enhanced pairwise CRF framework that improves semantic segmentation accuracy while reducing training complexity by leveraging superpixel cues and reusing existing pairwise model parameters.

Contribution

It proposes a novel superpixel-enhanced pairwise potential that incorporates higher-order cues with lower complexity, outperforming higher-order CRFs and reusing pre-trained models for better accuracy.

Findings

Outperforms higher-order CRFs in accuracy.

Reduces training complexity compared to higher-order models.

Effective on MSRC-21 and PASCAL VOC 2012 datasets.

Abstract

Superpixel-based Higher-order Conditional Random Fields (CRFs) are effective in enforcing long-range consistency in pixel-wise labeling problems, such as semantic segmentation. However, their major short coming is considerably longer time to learn higher-order potentials and extra hyperparameters and/or weights compared with pairwise models. This paper proposes a superpixel-enhanced pairwise CRF framework that consists of the conventional pairwise as well as our proposed superpixel-enhanced pairwise (SP-Pairwise) potentials. SP-Pairwise potentials incorporate the superpixel-based higher-order cues by conditioning on a segment filtered image and share the same set of parameters as the conventional pairwise potentials. Therefore, the proposed superpixel-enhanced pairwise CRF has a lower time complexity in parameter learning and at the same time it outperforms higher-order CRF in terms of inference accuracy. Moreover, the new scheme takes advantage of the pre-trained pairwise models by reusing their parameters and/or weights, which provides a significant accuracy boost on the basis of CRF-RNN even without training. Experiments on MSRC-21 and PASCAL VOC 2012 dataset confirm the effectiveness of our method.