On-the-Fly Learning in a Perpetual Learning Machine

TL;DR

This paper introduces a Perpetual Learning Machine, a brain-inspired deep neural network capable of continuous, on-the-fly learning and memory integration through self-supervised perpetual stochastic gradient descent, unifying learning and memory.

Contribution

It proposes a novel DNN architecture that enables dynamic, ongoing learning and memory retention, inspired by brain mechanisms, using a self-supervised perpetual learning process.

Findings

Demonstrates continuous, on-the-fly learning capability.

Unifies learning and memory in a single framework.

Explores the duality of abstraction and synthesis in deep learning.

Abstract

Despite the promise of brain-inspired machine learning, deep neural networks (DNN) have frustratingly failed to bridge the deceptively large gap between learning and memory. Here, we introduce a Perpetual Learning Machine; a new type of DNN that is capable of brain-like dynamic 'on the fly' learning because it exists in a self-supervised state of Perpetual Stochastic Gradient Descent. Thus, we provide the means to unify learning and memory within a machine learning framework. We also explore the elegant duality of abstraction and synthesis: the Yin and Yang of deep learning.