LightTS: Lightweight Time Series Classification with Adaptive Ensemble Distillation -- Extended Version

TL;DR

LightTS introduces an adaptive ensemble distillation method to create lightweight, resource-efficient time series classifiers that maintain high accuracy, making ensemble learning feasible for edge devices.

Contribution

The paper presents a novel adaptive ensemble distillation technique and Pareto optimal model selection for resource-constrained time series classification.

Findings

LightTS outperforms existing methods on 128 real-world datasets.

Adaptive weighting improves ensemble distillation effectiveness.

Pareto optimal models balance accuracy and size effectively.

Abstract

Due to the sweeping digitalization of processes, increasingly vast amounts of time series data are being produced. Accurate classification of such time series facilitates decision making in multiple domains. State-of-the-art classification accuracy is often achieved by ensemble learning where results are synthesized from multiple base models. This characteristic implies that ensemble learning needs substantial computing resources, preventing their use in resource-limited environments, such as in edge devices. To extend the applicability of ensemble learning, we propose the LightTS framework that compresses large ensembles into lightweight models while ensuring competitive accuracy. First, we propose adaptive ensemble distillation that assigns adaptive weights to different base models such that their varying classification capabilities contribute purposefully to the training of the lightweight model. Second, we propose means of identifying Pareto optimal settings w.r.t. model accuracy and model size, thus enabling users with a space budget to select the most accurate lightweight model. We report on experiments using 128 real-world time series sets and different types of base models that justify key decisions in the design of LightTS and provide evidence that LightTS is able to outperform competitors.