Synergy Between Semantic Segmentation and Image Denoising via Alternate Boosting

TL;DR

This paper introduces a novel deep learning model that alternates between image denoising and semantic segmentation, leveraging their synergy to improve both denoising quality and segmentation accuracy in noisy images.

Contribution

A new boosting network that performs denoising and segmentation alternately, utilizing semantic information to enhance denoising and vice versa.

Findings

Denoising quality is significantly improved.

Segmentation accuracy approaches that of clean images.

The method outperforms existing approaches in noisy conditions.

Abstract

The capability of image semantic segmentation may be deteriorated due to noisy input image, where image denoising prior to segmentation helps. Both image denoising and semantic segmentation have been developed significantly with the advance of deep learning. Thus, we are interested in the synergy between them by using a holistic deep model. We observe that not only denoising helps combat the drop of segmentation accuracy due to noise, but also pixel-wise semantic information boosts the capability of denoising. We then propose a boosting network to perform denoising and segmentation alternately. The proposed network is composed of multiple segmentation and denoising blocks (SDBs), each of which estimates semantic map then uses the map to regularize denoising. Experimental results show that the denoised image quality is improved substantially and the segmentation accuracy is improved to close to that of clean images. Our code and models will be made publicly available.