Investigating the feature collection for semantic segmentation via single skip connection

TL;DR

This paper systematically investigates the role and characteristics of skip connections in deep convolutional networks for semantic segmentation, aiming to identify the most effective features and configurations for improved performance.

Contribution

It provides an exhaustive analysis of skip connections in state-of-the-art networks and offers insights into optimal feature utilization for semantic segmentation.

Findings

Different skip connections have distinct feature properties.

Certain skip connections significantly improve segmentation accuracy.

Guidelines for using recent models in semantic segmentation are proposed.

Abstract

Since the study of deep convolutional neural network became prevalent, one of the important discoveries is that a feature map from a convolutional network can be extracted before going into the fully connected layer and can be used as a saliency map for object detection. Furthermore, the model can use features from each different layer for accurate object detection: the features from different layers can have different properties. As the model goes deeper, it has many latent skip connections and feature maps to elaborate object detection. Although there are many intermediate layers that we can use for semantic segmentation through skip connection, still the characteristics of each skip connection and the best skip connection for this task are uncertain. Therefore, in this study, we exhaustively research skip connections of state-of-the-art deep convolutional networks and investigate the characteristics of the features from each intermediate layer. In addition, this study would suggest how to use a recent deep neural network model for semantic segmentation and it would therefore become a cornerstone for later studies with the state-of-the-art network models.