U-TELL: Unsupervised Task Expert Lifelong Learning

TL;DR

U-TELL is an unsupervised lifelong learning model that uses task experts with autoencoders and clustering to learn sequential tasks without catastrophic forgetting, outperforming state-of-the-art methods on multiple benchmarks.

Contribution

The paper introduces U-TELL, a novel unsupervised lifelong learning framework with task experts, structured data generation, and task assignment, addressing catastrophic forgetting without sample replay.

Findings

U-TELL outperforms five SOTA unsupervised CL methods on seven benchmarks.

U-TELL trains over six times faster than the best baseline.

U-TELL effectively handles various CL scenarios with limited label information.

Abstract

Continual learning (CL) models are designed to learn new tasks arriving sequentially without re-training the network. However, real-world ML applications have very limited label information and these models suffer from catastrophic forgetting. To address these issues, we propose an unsupervised CL model with task experts called Unsupervised Task Expert Lifelong Learning (U-TELL) to continually learn the data arriving in a sequence addressing catastrophic forgetting. During training of U-TELL, we introduce a new expert on arrival of a new task. Our proposed architecture has task experts, a structured data generator and a task assigner. Each task expert is composed of 3 blocks; i) a variational autoencoder to capture the task distribution and perform data abstraction, ii) a k-means clustering module, and iii) a structure extractor to preserve latent task data signature. During testing, task assigner selects a suitable expert to perform clustering. U-TELL does not store or replay task samples, instead, we use generated structured samples to train the task assigner. We compared U-TELL with five SOTA unsupervised CL methods. U-TELL outperformed all baselines on seven benchmarks and one industry dataset for various CL scenarios with a training time over 6 times faster than the best performing baseline.