A Multi-Layer Approach to Superpixel-based Higher-order Conditional Random Field for Semantic Image Segmentation

TL;DR

This paper introduces a multi-layer CRF framework that simplifies higher-order superpixel-based CRFs, improving boundary delineation in semantic image segmentation while reducing computational complexity.

Contribution

The paper proposes a novel multi-layer approach that approximates higher-order potentials using pairwise CRFs, enabling effective segmentation without additional training.

Findings

Enhanced boundary delineation of object categories.

Reduced inference complexity compared to traditional higher-order CRFs.

Validated on MSRC-21 and PASCAL VOC 2012 datasets.

Abstract

Superpixel-based Higher-order Conditional random fields (SP-HO-CRFs) are known for their effectiveness in enforcing both short and long spatial contiguity for pixelwise labelling in computer vision. However, their higher-order potentials are usually too complex to learn and often incur a high computational cost in performing inference. We propose an new approximation approach to SP-HO-CRFs that resolves these problems. Our approach is a multi-layer CRF framework that inherits the simplicity from pairwise CRFs by formulating both the higher-order and pairwise cues into the same pairwise potentials in the first layer. Essentially, this approach provides accuracy enhancement on the basis of pairwise CRFs without training by reusing their pre-trained parameters and/or weights. The proposed multi-layer approach performs especially well in delineating the boundary details (boarders) of object categories such as "trees" and "bushes". Multiple sets of experiments conducted on dataset MSRC-21 and PASCAL VOC 2012 validate the effectiveness and efficiency of the proposed methods.