Hybrid Neural Network Architecture for On-Line Learning

TL;DR

This paper introduces a hybrid neural network architecture combining surface learning and deep learning components to enhance online learning performance, especially in chaotic time-series prediction and function approximation.

Contribution

It presents a novel hybrid neural network design that leverages the strengths of both quick adaptation and high accuracy, outperforming traditional models in specific tasks.

Findings

Hybrid architecture achieves lower RMS error than traditional networks.

Hybrid model adapts quickly to new data modes.

Superior performance in chaotic time-series prediction.

Abstract

Approaches to machine intelligence based on brain models have stressed the use of neural networks for generalization. Here we propose the use of a hybrid neural network architecture that uses two kind of neural networks simultaneously: (i) a surface learning agent that quickly adapt to new modes of operation; and, (ii) a deep learning agent that is very accurate within a specific regime of operation. The two networks of the hybrid architecture perform complementary functions that improve the overall performance. The performance of the hybrid architecture has been compared with that of back-propagation perceptrons and the CC and FC networks for chaotic time-series prediction, the CATS benchmark test, and smooth function approximation. It has been shown that the hybrid architecture provides a superior performance based on the RMS error criterion.