Technical Report: Combining knowledge from Transfer Learning during training and Wide Resnets

TL;DR

This paper combines Wide ResNet architecture and transfer learning principles to improve deep neural network performance by utilizing all layers as information sources and employing deeper layers instead of longer sequences.

Contribution

It introduces a novel architecture that integrates transfer learning ideas with Wide ResNets, using all layers for the last layer and emphasizing deeper layers for better results.

Findings

Improved accuracy with high data augmentation.

Better results with standard data augmentation.

Effective combination of transfer learning and Wide ResNet concepts.

Abstract

In this report, we combine the idea of Wide ResNets and transfer learning to optimize the architecture of deep neural networks. The first improvement of the architecture is the use of all layers as information source for the last layer. This idea comes from transfer learning, which uses networks pre-trained on other data and extracts different levels of the network as input for the new task. The second improvement is the use of deeper layers instead of deeper sequences of blocks. This idea comes from Wide ResNets. Using both optimizations, both high data augmentation and standard data augmentation can produce better results for different models. Link: https://github.com/wolfgangfuhl/PublicationStuff/tree/master/TechnicalReport1/Supp