User-friendly Foundation Model Adapters for Multivariate Time Series Classification

TL;DR

This paper proposes dimensionality reduction techniques, including PCA and neural adapters, to make foundation models for multivariate time series classification more efficient and accessible on standard GPUs, achieving significant speedups and scalability.

Contribution

It introduces a novel approach combining classical and neural methods to reduce data dimensionality, enabling resource-efficient foundation model deployment.

Findings

Up to 10x inference speedup without performance loss

Able to fit 4.5x more datasets on a single GPU

Demonstrates effective dimensionality reduction for time series data

Abstract

Foundation models, while highly effective, are often resource-intensive, requiring substantial inference time and memory. This paper addresses the challenge of making these models more accessible with limited computational resources by exploring dimensionality reduction techniques. Our goal is to enable users to run large pre-trained foundation models on standard GPUs without sacrificing performance. We investigate classical methods such as Principal Component Analysis alongside neural network-based adapters, aiming to reduce the dimensionality of multivariate time series data while preserving key features. Our experiments show up to a 10x speedup compared to the baseline model, without performance degradation, and enable up to 4.5x more datasets to fit on a single GPU, paving the way for more user-friendly and scalable foundation models.