Improving information retention in large scale online continual learning

TL;DR

This paper investigates the challenge of information retention in large-scale online continual learning, revealing limitations of naive SGD and proposing an adaptive moving average optimizer with a new learning rate schedule to improve performance.

Contribution

It introduces an adaptive moving average optimizer and a novel learning rate schedule specifically designed to enhance information retention in large-scale online continual learning.

Findings

AMA+MALR improves retention on benchmarks

Naive SGD fails to retain information long-term

Proposed methods outperform existing approaches

Abstract

Given a stream of data sampled from non-stationary distributions, online continual learning (OCL) aims to adapt efficiently to new data while retaining existing knowledge. The typical approach to address information retention (the ability to retain previous knowledge) is keeping a replay buffer of a fixed size and computing gradients using a mixture of new data and the replay buffer. Surprisingly, the recent work (Cai et al., 2021) suggests that information retention remains a problem in large scale OCL even when the replay buffer is unlimited, i.e., the gradients are computed using all past data. This paper focuses on this peculiarity to understand and address information retention. To pinpoint the source of this problem, we theoretically show that, given limited computation budgets at each time step, even without strict storage limit, naively applying SGD with constant or constantly decreasing learning rates fails to optimize information retention in the long term. We propose using a moving average family of methods to improve optimization for non-stationary objectives. Specifically, we design an adaptive moving average (AMA) optimizer and a moving-average-based learning rate schedule (MALR). We demonstrate the effectiveness of AMA+MALR on large-scale benchmarks, including Continual Localization (CLOC), Google Landmarks, and ImageNet. Code will be released upon publication.