Error Correction for Dense Semantic Image Labeling

TL;DR

This paper introduces a novel parallel architecture for dense semantic image labeling that combines label propagation, replacement, and fusion networks to improve accuracy and efficiency over existing methods.

Contribution

It proposes a new multi-network framework that leverages initial label estimates and context modeling to enhance dense semantic labeling performance.

Findings

Outperforms state-of-the-art on semantic segmentation and face parsing datasets.

Improves boundary accuracy through label propagation.

Reduces inference time compared to traditional probabilistic models.

Abstract

Pixelwise semantic image labeling is an important, yet challenging, task with many applications. Typical approaches to tackle this problem involve either the training of deep networks on vast amounts of images to directly infer the labels or the use of probabilistic graphical models to jointly model the dependencies of the input (i.e. images) and output (i.e. labels). Yet, the former approaches do not capture the structure of the output labels, which is crucial for the performance of dense labeling, and the latter rely on carefully hand-designed priors that require costly parameter tuning via optimization techniques, which in turn leads to long inference times. To alleviate these restrictions, we explore how to arrive at dense semantic pixel labels given both the input image and an initial estimate of the output labels. We propose a parallel architecture that: 1) exploits the context information through a LabelPropagation network to propagate correct labels from nearby pixels to improve the object boundaries, 2) uses a LabelReplacement network to directly replace possibly erroneous, initial labels with new ones, and 3) combines the different intermediate results via a Fusion network to obtain the final per-pixel label. We experimentally validate our approach on two different datasets for the semantic segmentation and face parsing tasks respectively, where we show improvements over the state-of-the-art. We also provide both a quantitative and qualitative analysis of the generated results.