Sketches for Time-Dependent Machine Learning

TL;DR

This paper introduces a method to improve time series machine learning models by maintaining and utilizing evolving statistical summaries of data, enhancing prediction accuracy without significant computational costs.

Contribution

It presents a novel approach to incorporate time-dependent data summaries into models, improving their adaptability to changing data distributions.

Findings

Significant performance improvements in classification tasks.

Efficient maintenance of statistical summaries with minimal overhead.

Enhanced model robustness to data distribution shifts.

Abstract

Time series data can be subject to changes in the underlying process that generates them and, because of these changes, models built on old samples can become obsolete or perform poorly. In this work, we present a way to incorporate information about the current data distribution and its evolution across time into machine learning algorithms. Our solution is based on efficiently maintaining statistics, particularly the mean and the variance, of data features at different time resolutions. These data summarisations can be performed over the input attributes, in which case they can then be fed into the model as additional input features, or over latent representations learned by models, such as those of Recurrent Neural Networks. In classification tasks, the proposed techniques can significantly outperform the prediction capabilities of equivalent architectures with no feature / latent summarisations. Furthermore, these modifications do not introduce notable computational and memory overhead when properly adjusted.