Online Resource Allocation for Edge Intelligence with Colocated Model Retraining and Inference

TL;DR

This paper presents ORRIC, an online algorithm for resource allocation that balances model retraining and inference on edge servers, improving accuracy amid data drift and resource constraints.

Contribution

Introduces ORRIC, a lightweight, explainable online approximation algorithm for resource allocation in colocated model retraining and inference at the edge.

Findings

ORRIC outperforms traditional inference-only methods in accuracy.

The algorithm adapts well to persistent data drift scenarios.

Experimental validation confirms effectiveness in real edge environments.

Abstract

With edge intelligence, AI models are increasingly pushed to the edge to serve ubiquitous users. However, due to the drift of model, data, and task, AI model deployed at the edge suffers from degraded accuracy in the inference serving phase. Model retraining handles such drifts by periodically retraining the model with newly arrived data. When colocating model retraining and model inference serving for the same model on resource-limited edge servers, a fundamental challenge arises in balancing the resource allocation for model retraining and inference, aiming to maximize long-term inference accuracy. This problem is particularly difficult due to the underlying mathematical formulation being time-coupled, non-convex, and NP-hard. To address these challenges, we introduce a lightweight and explainable online approximation algorithm, named ORRIC, designed to optimize resource allocation for adaptively balancing the accuracy of model training and inference. The competitive ratio of ORRIC outperforms that of the traditional Inference-Only paradigm, especially when data drift persists for a sufficiently lengthy time. This highlights the advantages and applicable scenarios of colocating model retraining and inference. Notably, ORRIC can be translated into several heuristic algorithms for different resource environments. Experiments conducted in real scenarios validate the effectiveness of ORRIC.