Sequence Transferability and Task Order Selection in Continual Learning

TL;DR

This paper explores how task sequence properties affect continual learning performance, introducing measures for transferability and a new task order selection method that outperforms random strategies.

Contribution

It proposes two novel transferability measures and a new task order selection method tailored for continual learning, improving upon traditional random selection.

Findings

The proposed measures effectively quantify sequence transferability.

The new task order selection method improves learning accuracy.

Empirical results show better performance than random task selection.

Abstract

In continual learning, understanding the properties of task sequences and their relationships to model performance is important for developing advanced algorithms with better accuracy. However, efforts in this direction remain underdeveloped despite encouraging progress in methodology development. In this work, we investigate the impacts of sequence transferability on continual learning and propose two novel measures that capture the total transferability of a task sequence, either in the forward or backward direction. Based on the empirical properties of these measures, we then develop a new method for the task order selection problem in continual learning. Our method can be shown to offer a better performance than the conventional strategy of random task selection.