Fine Tuning Method by using Knowledge Acquisition from Deep Belief Network

TL;DR

This paper introduces an adaptive learning method for Deep Belief Networks that self-organizes neurons based on input patterns, embedding knowledge to improve classification accuracy on unknown data.

Contribution

It proposes a novel adaptive structure learning method for RBMs and DBNs that enhances classification performance by incorporating knowledge about data patterns.

Findings

Achieved 97.1% accuracy on unknown data set

Improved classification of patterns with embedded knowledge

Demonstrated effectiveness on CIFAR-10 benchmark

Abstract

We developed an adaptive structure learning method of Restricted Boltzmann Machine (RBM) which can generate/annihilate neurons by self-organizing learning method according to input patterns. Moreover, the adaptive Deep Belief Network (DBN) in the assemble process of pre-trained RBM layer was developed. The proposed method presents to score a great success to the training data set for big data benchmark test such as CIFAR-10. However, the classification capability of the test data set, which are included unknown patterns, is high, but does not lead perfect correct solution. We investigated the wrong specified data and then some characteristic patterns were found. In this paper, the knowledge related to the patterns is embedded into the classification algorithm of trained DBN. As a result, the classification capability can achieve a great success (97.1\% to unknown data set).