A Novel Approach to Lifelong Learning: The Plastic Support Structure

TL;DR

This paper introduces the Plastic Support Structure (PSS), a compact, encapsulated method for lifelong learning that allows neural networks to expand capacity for new tasks without forgetting previous ones, with improved efficiency and interpretability.

Contribution

The paper presents the PSS, a novel, task-specific support structure enabling neural networks to learn new tasks incrementally without prior task knowledge, reducing parameters and enhancing interpretability.

Findings

PSS performs comparably to existing methods on public datasets.

PSS requires fewer parameters in some cases.

PSS offers a more understandable, encapsulated approach for lifelong learning.

Abstract

We propose a novel approach to lifelong learning, introducing a compact encapsulated support structure which endows a network with the capability to expand its capacity as needed to learn new tasks while preventing the loss of learned tasks. This is achieved by splitting neurons with high semantic drift and constructing an adjacent network to encode the new tasks at hand. We call this the Plastic Support Structure (PSS), it is a compact structure to learn new tasks that cannot be efficiently encoded in the existing structure of the network. We validate the PSS on public datasets against existing lifelong learning architectures, showing it performs similarly to them but without prior knowledge of the task and in some cases with fewer parameters and in a more understandable fashion where the PSS is an encapsulated container for specific features related to specific tasks, thus making it an ideal "add-on" solution for endowing a network to learn more tasks.