Lightweight Conditional Model Extrapolation for Streaming Data under Class-Prior Shift

TL;DR

LIMES is a lightweight meta-learning based method for adapting classifiers to non-stationary streaming data with class-prior shift, achieving higher accuracy with minimal overhead.

Contribution

We propose LIMES, a novel meta-learning approach that enables efficient classifier adaptation for streaming data with class-prior shift, requiring no additional trainable parameters.

Findings

LIMES outperforms alternative methods in accuracy on Twitter data tasks.

The method effectively predicts adaptation parameters for future data distributions.

LIMES maintains low computational and memory overhead.

Abstract

We introduce LIMES, a new method for learning with non-stationary streaming data, inspired by the recent success of meta-learning. The main idea is not to attempt to learn a single classifier that would have to work well across all occurring data distributions, nor many separate classifiers, but to exploit a hybrid strategy: we learn a single set of model parameters from which a specific classifier for any specific data distribution is derived via classifier adaptation. Assuming a multi-class classification setting with class-prior shift, the adaptation step can be performed analytically with only the classifier's bias terms being affected. Another contribution of our work is an extrapolation step that predicts suitable adaptation parameters for future time steps based on the previous data. In combination, we obtain a lightweight procedure for learning from streaming data with varying class distribution that adds no trainable parameters and almost no memory or computational overhead compared to training a single model. Experiments on a set of exemplary tasks using Twitter data show that LIMES achieves higher accuracy than alternative approaches, especially with respect to the relevant real-world metric of lowest within-day accuracy.