LEAF: Navigating Concept Drift in Cellular Networks

TL;DR

This paper addresses the challenge of concept drift in cellular network machine learning models by characterizing its occurrence, demonstrating the limitations of frequent retraining, and proposing a novel mitigation method called LEAF that outperforms standard approaches.

Contribution

The paper introduces LEAF, a new methodology for detecting, explaining, and mitigating concept drift in cellular networks, improving model stability and reducing retraining costs.

Findings

Concept drift occurs across many KPIs regardless of model or data size.

Frequent retraining alone does not effectively mitigate drift and can degrade accuracy.

LEAF outperforms standard retraining methods in real-world cellular network data.

Abstract

Operational networks commonly rely on machine learning models for many tasks, including detecting anomalies, inferring application performance, and forecasting demand. Yet, model accuracy can degrade due to concept drift, whereby the relationship between the features and the target to be predicted changes. Mitigating concept drift is an essential part of operationalizing machine learning models in general, but is of particular importance in networking's highly dynamic deployment environments. In this paper, we first characterize concept drift in a large cellular network for a major metropolitan area in the United States. We find that concept drift occurs across many important key performance indicators (KPIs), independently of the model, training set size, and time interval -- thus necessitating practical approaches to detect, explain, and mitigate it. We then show that frequent model retraining with newly available data is not sufficient to mitigate concept drift, and can even degrade model accuracy further. Finally, we develop a new methodology for concept drift mitigation, Local Error Approximation of Features (LEAF). LEAF works by detecting drift; explaining the features and time intervals that contribute the most to drift; and mitigates it using forgetting and over-sampling. We evaluate LEAF against industry-standard mitigation approaches (notably, periodic retraining) with more than four years of cellular KPI data. Our initial tests with a major cellular provider in the US show that LEAF consistently outperforms periodic and triggered retraining on complex, real-world data while reducing costly retraining operations.