Online Analytic Exemplar-Free Continual Learning with Large Models for Imbalanced Autonomous Driving Task

TL;DR

This paper introduces AEF-OCL, an exemplar-free online continual learning method using analytic solutions and a pseudo-feature generator, effectively addressing catastrophic forgetting and data imbalance in autonomous driving scenarios.

Contribution

The paper proposes a novel exemplar-free continual learning algorithm with an analytic ridge regression approach and a pseudo-feature generator for imbalanced autonomous driving tasks.

Findings

Outperforms existing methods on SODA10M dataset

Effectively mitigates catastrophic forgetting without exemplars

Addresses data imbalance with pseudo-feature over-sampling

Abstract

In autonomous driving, even a meticulously trained model can encounter failures when facing unfamiliar scenarios. One of these scenarios can be formulated as an online continual learning (OCL) problem. That is, data come in an online fashion, and models are updated according to these streaming data. Two major OCL challenges are catastrophic forgetting and data imbalance. To address these challenges, in this paper, we propose an Analytic Exemplar-Free Online Continual Learning algorithm (AEF-OCL). The AEF-OCL leverages analytic continual learning principles and employs ridge regression as a classifier for features extracted by a large backbone network. It solves the OCL problem by recursively calculating the analytical solution, ensuring an equalization between the continual learning and its joint-learning counterpart, and works without the need to save any used samples (i.e., exemplar-free). Additionally, we introduce a Pseudo-Features Generator (PFG) module that recursively estimates the mean and the variance of real features for each class. It over-samples offset pseudo-features from the same normal distribution as the real features, thereby addressing the data imbalance issue. Experimental results demonstrate that despite being an exemplar-free strategy, our method outperforms various methods on the autonomous driving SODA10M dataset. Source code is available at https://github.com/ZHUANGHP/Analytic-continual-learning.