Utility-based Perturbed Gradient Descent: An Optimizer for Continual Learning

TL;DR

This paper introduces UPGD, an online optimization algorithm designed for continual learning that mitigates catastrophic forgetting by selectively perturbing weights based on their utility, thereby improving adaptability and retention.

Contribution

The paper proposes UPGD, a novel utility-based perturbed gradient descent algorithm that enhances continual learning by balancing plasticity and stability.

Findings

UPGD reduces catastrophic forgetting in continual learning tasks.

UPGD maintains higher plasticity compared to traditional methods.

Empirical results demonstrate improved performance of representation learning methods with UPGD.

Abstract

Modern representation learning methods often struggle to adapt quickly under non-stationarity because they suffer from catastrophic forgetting and decaying plasticity. Such problems prevent learners from fast adaptation since they may forget useful features or have difficulty learning new ones. Hence, these methods are rendered ineffective for continual learning. This paper proposes Utility-based Perturbed Gradient Descent (UPGD), an online learning algorithm well-suited for continual learning agents. UPGD protects useful weights or features from forgetting and perturbs less useful ones based on their utilities. Our empirical results show that UPGD helps reduce forgetting and maintain plasticity, enabling modern representation learning methods to work effectively in continual learning.