Adaptive Learning Method of Recurrent Temporal Deep Belief Network to Analyze Time Series Data

TL;DR

This paper introduces an adaptive learning method for recurrent temporal deep belief networks that automatically optimizes network structure, improving classification performance on time series data.

Contribution

It extends an adaptive learning approach to recurrent temporal RBMs and DBNs, enabling self-organization of network structure during training.

Findings

Higher classification accuracy than conventional methods

Effective self-organization of network layers and neurons

Improved modeling of time series data

Abstract

Deep Learning has the hierarchical network architecture to represent the complicated features of input patterns. Such architecture is well known to represent higher learning capability compared with some conventional models if the best set of parameters in the optimal network structure is found. We have been developing the adaptive learning method that can discover the optimal network structure in Deep Belief Network (DBN). The learning method can construct the network structure with the optimal number of hidden neurons in each Restricted Boltzmann Machine and with the optimal number of layers in the DBN during learning phase. The network structure of the learning method can be self-organized according to given input patterns of big data set. In this paper, we embed the adaptive learning method into the recurrent temporal RBM and the self-generated layer into DBN. In order to verify the effectiveness of our proposed method, the experimental results are higher classification capability than the conventional methods in this paper.