Learning what to remember

TL;DR

This paper models lifelong learning as an online decision problem where a learner must efficiently decide which facts to remember from a continuous stream, balancing memory constraints and accuracy.

Contribution

It introduces a new mathematical framework for memory-limited lifelong learning and proposes an alternative algorithm with near-optimal regret guarantees.

Findings

Designed an alternative scheme with close-to-optimal regret guarantees

Identified limitations of the multiplicative weights update algorithm in this context

Provided a formal model for memory-constrained lifelong learning

Abstract

We consider a lifelong learning scenario in which a learner faces a neverending and arbitrary stream of facts and has to decide which ones to retain in its limited memory. We introduce a mathematical model based on the online learning framework, in which the learner measures itself against a collection of experts that are also memory-constrained and that reflect different policies for what to remember. Interspersed with the stream of facts are occasional questions, and on each of these the learner incurs a loss if it has not remembered the corresponding fact. Its goal is to do almost as well as the best expert in hindsight, while using roughly the same amount of memory. We identify difficulties with using the multiplicative weights update algorithm in this memory-constrained scenario, and design an alternative scheme whose regret guarantees are close to the best possible.