AMUSE: Adaptive Model Updating using a Simulated Environment

TL;DR

AMUSE is a reinforcement learning-based method that learns optimal model update timings in the presence of concept drift by training in a simulated environment, balancing performance and update costs.

Contribution

It introduces a novel reinforcement learning approach for adaptive model updating using a simulated environment to anticipate concept drift.

Findings

AMUSE outperforms traditional update strategies in simulated scenarios.

The method effectively balances update costs with classifier performance.

Empirical results demonstrate improved adaptation to concept drift.

Abstract

Prediction models frequently face the challenge of concept drift, in which the underlying data distribution changes over time, weakening performance. Examples can include models which predict loan default, or those used in healthcare contexts. Typical management strategies involve regular model updates or updates triggered by concept drift detection. However, these simple policies do not necessarily balance the cost of model updating with improved classifier performance. We present AMUSE (Adaptive Model Updating using a Simulated Environment), a novel method leveraging reinforcement learning trained within a simulated data generating environment, to determine update timings for classifiers. The optimal updating policy depends on the current data generating process and ongoing drift process. Our key idea is that we can train an arbitrarily complex model updating policy by creating a training environment in which possible episodes of drift are simulated by a parametric model, which represents expectations of possible drift patterns. As a result, AMUSE proactively recommends updates based on estimated performance improvements, learning a policy that balances maintaining model performance with minimizing update costs. Empirical results confirm the effectiveness of AMUSE in simulated data.