Cost-effective On-device Continual Learning over Memory Hierarchy with Miro

TL;DR

Miro is a system that enhances on-device continual learning by dynamically optimizing memory replay configurations based on resource states, significantly improving cost-effectiveness on energy-sensitive edge devices.

Contribution

This work introduces Miro, the first system to adaptively configure hierarchical memory replay for cost-effective continual learning on edge devices.

Findings

Miro achieves higher cost-effectiveness than baseline systems.

Miro adapts configurations with low overhead.

Extensive evaluations confirm Miro's superior performance.

Abstract

Continual learning (CL) trains NN models incrementally from a continuous stream of tasks. To remember previously learned knowledge, prior studies store old samples over a memory hierarchy and replay them when new tasks arrive. Edge devices that adopt CL to preserve data privacy are typically energy-sensitive and thus require high model accuracy while not compromising energy efficiency, i.e., cost-effectiveness. Our work is the first to explore the design space of hierarchical memory replay-based CL to gain insights into achieving cost-effectiveness on edge devices. We present Miro, a novel system runtime that carefully integrates our insights into the CL framework by enabling it to dynamically configure the CL system based on resource states for the best cost-effectiveness. To reach this goal, Miro also performs online profiling on parameters with clear accuracy-energy trade-offs and adapts to optimal values with low overhead. Extensive evaluations show that Miro significantly outperforms baseline systems we build for comparison, consistently achieving higher cost-effectiveness.