Semantic Image Segmentation via Deep Parsing Network

TL;DR

This paper introduces Deep Parsing Network (DPN), a CNN-based approach for semantic image segmentation that models complex MRF relations in a single forward pass, achieving state-of-the-art results efficiently.

Contribution

The paper presents DPN, a novel CNN architecture that models high-order relations and label contexts in MRFs for end-to-end semantic segmentation, simplifying and accelerating previous methods.

Findings

Achieves state-of-the-art accuracy on PASCAL VOC 2012

Models complex MRF relations within a single network pass

Speeds up inference through GPU parallelization

Abstract

This paper addresses semantic image segmentation by incorporating rich information into Markov Random Field (MRF), including high-order relations and mixture of label contexts. Unlike previous works that optimized MRFs using iterative algorithm, we solve MRF by proposing a Convolutional Neural Network (CNN), namely Deep Parsing Network (DPN), which enables deterministic end-to-end computation in a single forward pass. Specifically, DPN extends a contemporary CNN architecture to model unary terms and additional layers are carefully devised to approximate the mean field algorithm (MF) for pairwise terms. It has several appealing properties. First, different from the recent works that combined CNN and MRF, where many iterations of MF were required for each training image during back-propagation, DPN is able to achieve high performance by approximating one iteration of MF. Second, DPN represents various types of pairwise terms, making many existing works as its special cases. Third, DPN makes MF easier to be parallelized and speeded up in Graphical Processing Unit (GPU). DPN is thoroughly evaluated on the PASCAL VOC 2012 dataset, where a single DPN model yields a new state-of-the-art segmentation accuracy.