Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation

TL;DR

This paper introduces DeepLabv3+ which combines atrous spatial pyramid pooling with a decoder for improved semantic segmentation, achieving state-of-the-art results on PASCAL VOC 2012 and Cityscapes datasets.

Contribution

It proposes an enhanced encoder-decoder architecture that integrates atrous separable convolutions and a simple decoder to refine segmentation boundaries.

Findings

Achieves 89.0% mIOU on PASCAL VOC 2012

Achieves 82.1% mIOU on Cityscapes

Faster and stronger model due to depthwise separable convolutions

Abstract

Spatial pyramid pooling module or encode-decoder structure are used in deep neural networks for semantic segmentation task. The former networks are able to encode multi-scale contextual information by probing the incoming features with filters or pooling operations at multiple rates and multiple effective fields-of-view, while the latter networks can capture sharper object boundaries by gradually recovering the spatial information. In this work, we propose to combine the advantages from both methods. Specifically, our proposed model, DeepLabv3+, extends DeepLabv3 by adding a simple yet effective decoder module to refine the segmentation results especially along object boundaries. We further explore the Xception model and apply the depthwise separable convolution to both Atrous Spatial Pyramid Pooling and decoder modules, resulting in a faster and stronger encoder-decoder network. We demonstrate the effectiveness of the proposed model on PASCAL VOC 2012 and Cityscapes datasets, achieving the test set performance of 89.0\% and 82.1\% without any post-processing. Our paper is accompanied with a publicly available reference implementation of the proposed models in Tensorflow at \url{https://github.com/tensorflow/models/tree/master/research/deeplab}.