Liquid Ensemble Selection for Continual Learning

TL;DR

This paper introduces a delegation-based ensemble selection method for continual learning, enabling models to adaptively choose which members learn or predict, thereby improving accuracy and reducing forgetting under shifting data distributions.

Contribution

It proposes a novel delegation algorithm for dynamic ensemble member selection in continual learning, enhancing performance over naive approaches.

Findings

Delegation significantly improves accuracy during distribution shifts.

Dynamic ensemble selection reduces forgetting compared to static models.

Various delegation methods were evaluated, showing consistent performance gains.

Abstract

Continual learning aims to enable machine learning models to continually learn from a shifting data distribution without forgetting what has already been learned. Such shifting distributions can be broken into disjoint subsets of related examples; by training each member of an ensemble on a different subset it is possible for the ensemble as a whole to achieve much higher accuracy with less forgetting than a naive model. We address the problem of selecting which models within an ensemble should learn on any given data, and which should predict. By drawing on work from delegative voting we develop an algorithm for using delegation to dynamically select which models in an ensemble are active. We explore a variety of delegation methods and performance metrics, ultimately finding that delegation is able to provide a significant performance boost over naive learning in the face of distribution shifts.