CRF Learning with CNN Features for Image Segmentation

TL;DR

This paper introduces a novel image segmentation method combining CNN-derived deep features with CRF models, utilizing SSVM for parameter learning and incorporating spatial co-occurrence potentials for improved accuracy.

Contribution

It proposes integrating pre-trained CNN features into CRF models for image segmentation and introduces spatial co-occurrence potentials to enhance contextual labeling.

Findings

Outperforms existing methods on multiple benchmarks

Provides new baseline results for several datasets

Demonstrates effectiveness of CNN features in CRF-based segmentation

Abstract

Conditional Random Rields (CRF) have been widely applied in image segmentations. While most studies rely on hand-crafted features, we here propose to exploit a pre-trained large convolutional neural network (CNN) to generate deep features for CRF learning. The deep CNN is trained on the ImageNet dataset and transferred to image segmentations here for constructing potentials of superpixels. Then the CRF parameters are learnt using a structured support vector machine (SSVM). To fully exploit context information in inference, we construct spatially related co-occurrence pairwise potentials and incorporate them into the energy function. This prefers labelling of object pairs that frequently co-occur in a certain spatial layout and at the same time avoids implausible labellings during the inference. Extensive experiments on binary and multi-class segmentation benchmarks demonstrate the promise of the proposed method. We thus provide new baselines for the segmentation performance on the Weizmann horse, Graz-02, MSRC-21, Stanford Background and PASCAL VOC 2011 datasets.