Training Deeper Convolutional Networks with Deep Supervision

TL;DR

This paper introduces deep supervision in convolutional networks by adding auxiliary branches at intermediate layers, simplifying training and improving accuracy on large-scale image datasets.

Contribution

It proposes a practical method for training deeper CNNs using auxiliary supervision, enabling easier optimization and better performance.

Findings

Improved training efficiency for deep CNNs.

Achieved higher accuracy on ImageNet and MIT Places datasets.

Demonstrated effectiveness of deep supervision in deep learning models.

Abstract

One of the most promising ways of improving the performance of deep convolutional neural networks is by increasing the number of convolutional layers. However, adding layers makes training more difficult and computationally expensive. In order to train deeper networks, we propose to add auxiliary supervision branches after certain intermediate layers during training. We formulate a simple rule of thumb to determine where these branches should be added. The resulting deeply supervised structure makes the training much easier and also produces better classification results on ImageNet and the recently released, larger MIT Places dataset