On Continual Model Refinement in Out-of-Distribution Data Streams

TL;DR

This paper introduces a new continual learning problem called continual model refinement (CMR) for NLP models to adapt to out-of-distribution data streams, addressing realistic challenges like non-stationarity and error correction.

Contribution

The paper formulates the CMR problem, extends existing CL methods to this setting, and provides benchmarking tools including data stream algorithms and performance metrics.

Findings

Existing CL methods show promise but face challenges in CMR.

The proposed benchmarking suite enables systematic evaluation of models in dynamic OOD streams.

Studying CMR can improve the longevity of NLP models in real-world applications.

Abstract

Real-world natural language processing (NLP) models need to be continually updated to fix the prediction errors in out-of-distribution (OOD) data streams while overcoming catastrophic forgetting. However, existing continual learning (CL) problem setups cannot cover such a realistic and complex scenario. In response to this, we propose a new CL problem formulation dubbed continual model refinement (CMR). Compared to prior CL settings, CMR is more practical and introduces unique challenges (boundary-agnostic and non-stationary distribution shift, diverse mixtures of multiple OOD data clusters, error-centric streams, etc.). We extend several existing CL approaches to the CMR setting and evaluate them extensively. For benchmarking and analysis, we propose a general sampling algorithm to obtain dynamic OOD data streams with controllable non-stationarity, as well as a suite of metrics measuring various aspects of online performance. Our experiments and detailed analysis reveal the promise and challenges of the CMR problem, supporting that studying CMR in dynamic OOD streams can benefit the longevity of deployed NLP models in production.