Gated Feedback Refinement Network for Coarse-to-Fine Dense Semantic Image Labeling

TL;DR

This paper introduces Gated Feedback Refinement Network (G-FRNet), a novel deep learning architecture that progressively refines coarse segmentation predictions by effectively integrating local and global context, achieving state-of-the-art results.

Contribution

The paper proposes G-FRNet with gate units for better information passing, improving refinement in semantic segmentation over previous coarse-to-fine models.

Findings

Achieves state-of-the-art results on CamVid and Horse-Cow Parsing datasets.

Produces competitive results on PASCAL VOC 2012, PASCAL-Person-Part, and SUN-RGBD.

Demonstrates effective integration of local and global context through gating mechanisms.

Abstract

Effective integration of local and global contextual information is crucial for semantic segmentation and dense image labeling. We develop two encoder-decoder based deep learning architectures to address this problem. We first propose a network architecture called Label Refinement Network (LRN) that predicts segmentation labels in a coarse-to-fine fashion at several spatial resolutions. In this network, we also define loss functions at several stages to provide supervision at different stages of training. However, there are limits to the quality of refinement possible if ambiguous information is passed forward. In order to address this issue, we also propose Gated Feedback Refinement Network (G-FRNet) that addresses this limitation. Initially, G-FRNet makes a coarse-grained prediction which it progressively refines to recover details by effectively integrating local and global contextual information during the refinement stages. This is achieved by gate units proposed in this work, that control information passed forward in order to resolve the ambiguity. Experiments were conducted on four challenging dense labeling datasets (CamVid, PASCAL VOC 2012, Horse-Cow Parsing, PASCAL-Person-Part, and SUN-RGBD). G-FRNet achieves state-of-the-art semantic segmentation results on the CamVid and Horse-Cow Parsing datasets and produces results competitive with the best performing approaches that appear in the literature for the other three datasets.