Block Neural Network Avoids Catastrophic Forgetting When Learning Multiple Task

TL;DR

This paper introduces a deep feedforward neural network architecture capable of sequential learning across multiple tasks while avoiding catastrophic forgetting, reusing features, and requiring fewer resources and data.

Contribution

The proposed architecture enables sequential task learning with feature reuse and resource efficiency, addressing catastrophic forgetting in neural networks.

Findings

Successfully avoids catastrophic forgetting in sequential learning

Reuses features learned from previous tasks effectively

Requires fewer neurons, connections, and data for new tasks

Abstract

In the present work we propose a Deep Feed Forward network architecture which can be trained according to a sequential learning paradigm, where tasks of increasing difficulty are learned sequentially, yet avoiding catastrophic forgetting. The proposed architecture can re-use the features learned on previous tasks in a new task when the old tasks and the new one are related. The architecture needs fewer computational resources (neurons and connections) and less data for learning the new task than a network trained from scratch