Beyond Forward Shortcuts: Fully Convolutional Master-Slave Networks (MSNets) with Backward Skip Connections for Semantic Segmentation

TL;DR

This paper introduces a novel fully convolutional network architecture with backward skip connections from high to low layers, enhancing semantic segmentation performance by allowing high-level context to inform low-level feature extraction.

Contribution

The paper proposes a new Master-Slave network model with backward skip connections, a concept that counters traditional feed-forward design, improving feature integration for segmentation.

Findings

Improved segmentation accuracy on ADE20K, PASCAL-Context, and PASCAL VOC 2011 datasets.

Demonstrates the effectiveness of backward skip connections in deep CNNs.

Validates the proposed model's superiority over traditional forward-only networks.

Abstract

Recent deep CNNs contain forward shortcut connections; i.e. skip connections from low to high layers. Reusing features from lower layers that have higher resolution (location information) benefit higher layers to recover lost details and mitigate information degradation. However, during inference the lower layers do not know about high layer features, although they contain contextual high semantics that benefit low layers to adaptively extract informative features for later layers. In this paper, we study the influence of backward skip connections which are in the opposite direction to forward shortcuts, i.e. paths from high layers to low layers. To achieve this -- which indeed runs counter to the nature of feed-forward networks -- we propose a new fully convolutional model that consists of a pair of networks. A `Slave' network is dedicated to provide the backward connections from its top layers to the `Master' network's bottom layers. The Master network is used to produce the final label predictions. In our experiments we validate the proposed FCN model on ADE20K (ImageNet scene parsing), PASCAL-Context, and PASCAL VOC 2011 datasets.