Task agnostic continual learning with Pairwise layer architecture

TL;DR

This paper introduces a task-agnostic continual learning method using a pairwise interaction layer that enhances performance without relying on memory replay or task boundary information.

Contribution

It proposes a novel static architecture with a pairwise interaction layer that improves continual learning performance without task-specific mechanisms.

Findings

Achieves competitive results on MNIST and FashionMNIST.

Operates effectively in online streaming scenarios without task labels.

Does not require memory replay or task boundary detection.

Abstract

Most of the dominant approaches to continual learning are based on either memory replay, parameter isolation, or regularization techniques that require task boundaries to calculate task statistics. We propose a static architecture-based method that doesn't use any of these. We show that we can improve the continual learning performance by replacing the final layer of our networks with our pairwise interaction layer. The pairwise interaction layer uses sparse representations from a Winner-take-all style activation function to find the relevant correlations in the hidden layer representations. The networks using this architecture show competitive performance in MNIST and FashionMNIST-based continual image classification experiments. We demonstrate this in an online streaming continual learning setup where the learning system cannot access task labels or boundaries.