Evidential fully convolutional network for semantic segmentation

TL;DR

This paper introduces an evidential fully convolutional network (E-FCN) that combines deep learning with Dempster-Shafer theory to improve semantic segmentation accuracy and handle ambiguous pixels more effectively.

Contribution

The paper presents a novel hybrid architecture integrating a Dempster-Shafer layer with FCN, enabling joint training and better handling of uncertain classifications.

Findings

Improved segmentation accuracy on Pascal VOC 2011, MIT-scene Parsing, and SIFT Flow datasets.

Enhanced calibration and uncertainty estimation in semantic segmentation.

Effective imprecise classification of ambiguous pixels and outliers.

Abstract

We propose a hybrid architecture composed of a fully convolutional network (FCN) and a Dempster-Shafer layer for image semantic segmentation. In the so-called evidential FCN (E-FCN), an encoder-decoder architecture first extracts pixel-wise feature maps from an input image. A Dempster-Shafer layer then computes mass functions at each pixel location based on distances to prototypes. Finally, a utility layer performs semantic segmentation from mass functions and allows for imprecise classification of ambiguous pixels and outliers. We propose an end-to-end learning strategy for jointly updating the network parameters, which can make use of soft (imprecise) labels. Experiments using three databases (Pascal VOC 2011, MIT-scene Parsing and SIFT Flow) show that the proposed combination improves the accuracy and calibration of semantic segmentation by assigning confusing pixels to multi-class sets.