Residual CNDS

TL;DR

This paper investigates the impact of integrating residual connections into CNDS networks, demonstrating improved accuracy in image classification tasks by combining deep supervision with residual learning techniques.

Contribution

It introduces residual connections into CNDS networks, enhancing their accuracy and training efficiency for deep CNNs.

Findings

Residual connections improve CNDS accuracy

Enhanced training stability with residual CNDS

Better performance on image classification benchmarks

Abstract

Convolutional Neural networks nowadays are of tremendous importance for any image classification system. One of the most investigated methods to increase the accuracy of CNN is by increasing the depth of CNN. Increasing the depth by stacking more layers also increases the difficulty of training besides making it computationally expensive. Some research found that adding auxiliary forks after intermediate layers increases the accuracy. Specifying which intermediate layer shoud have the fork just addressed recently. Where a simple rule were used to detect the position of intermediate layers that needs the auxiliary supervision fork. This technique known as convolutional neural networks with deep supervision (CNDS). This technique enhanced the accuracy of classification over the straight forward CNN used on the MIT places dataset and ImageNet. In the other side, Residual Learning is another technique emerged recently to ease the training of very deep CNN. Residual Learning framwork changed the learning of layers from unreferenced functions to learning residual function with regard to the layer's input. Residual Learning achieved state of arts results on ImageNet 2015 and COCO competitions. In this paper, we study the effect of adding residual connections to CNDS network. Our experiments results show increasing of accuracy over using CNDS only.