Domain Adaptation and Transfer Learning in StochasticNets

TL;DR

This paper investigates how transfer learning can enhance StochasticNets, a neural network model with sparse connectivity, demonstrating a 7% performance improvement with transfer learning applied.

Contribution

It is the first study to evaluate transfer learning's effectiveness on StochasticNets, showing significant performance gains.

Findings

7% performance improvement with transfer learning

Transfer learning reduces training data requirements for StochasticNets

Demonstrates the viability of transfer learning in sparse neural network frameworks

Abstract

Transfer learning is a recent field of machine learning research that aims to resolve the challenge of dealing with insufficient training data in the domain of interest. This is a particular issue with traditional deep neural networks where a large amount of training data is needed. Recently, StochasticNets was proposed to take advantage of sparse connectivity in order to decrease the number of parameters that needs to be learned, which in turn may relax training data size requirements. In this paper, we study the efficacy of transfer learning on StochasticNet frameworks. Experimental results show ~7% improvement on StochasticNet performance when the transfer learning is applied in training step.