Capacity-Constrained Continual Learning

TL;DR

This paper investigates how agents with limited memory and computational resources can optimally allocate their capacity in continual learning scenarios, providing a theoretical foundation for resource management under constraints.

Contribution

It introduces a formal analysis of capacity-constrained continual learning using the LQG problem and derives optimal resource allocation strategies for such agents.

Findings

Derived a solution for the capacity-constrained LQG prediction problem.

Demonstrated optimal capacity allocation across decomposable sub-problems.

Provided a theoretical framework for learning under resource constraints.

Abstract

Any agents we can possibly build are subject to capacity constraints, as memory and compute resources are inherently finite. However, comparatively little attention has been dedicated to understanding how agents with limited capacity should allocate their resources for optimal performance. The goal of this paper is to shed some light on this question by studying a simple yet relevant continual learning problem: the capacity-constrained linear-quadratic-Gaussian (LQG) sequential prediction problem. We derive a solution to this problem under appropriate technical conditions. Moreover, for problems that can be decomposed into a set of sub-problems, we also demonstrate how to optimally allocate capacity across these sub-problems in the steady state. We view the results of this paper as a first step in the systematic theoretical study of learning under capacity constraints.