There is No "apple" in Timeseries: Rethinking TSFM through the Lens of Invariance

TL;DR

This paper argues that the limited concept coverage in timeseries datasets hampers the effectiveness of foundation models, advocating for a principled approach to dataset design based on invariance principles to improve generalization.

Contribution

It introduces a new perspective on dataset construction for timeseries models, emphasizing invariance coverage based on first principles for better model generalization.

Findings

Current datasets lack concept diversity like 'apple' in images

Naive data collection strategies fail to capture essential invariances

Principled dataset design can enhance timeseries model capabilities

Abstract

Timeseries foundation models (TSFMs) have multiplied, yet lightweight supervised baselines and even classical models often match them. We argue this gap stems from the naive importation of NLP or CV pipelines. In language and vision, large web-scale corpora densely capture human concepts i.e. there are countless images and text of apples. In contrast, timeseries data is built to complement the image and text modalities. There are no timeseries dataset that contains the concept apple. As a result, the scrape-everything-online paradigm fails for TS. We posit that progress demands a shift from opportunistic aggregation to principled design: constructing datasets that systematically span the space of invariance that preserve temporal semantics. To this end, we suggest that the ontology of timeseries invariances should be built based on first principles. Only by ensuring representational completeness through invariance coverage can TSFMs achieve the aligned structure necessary for generalisation, reasoning, and truly emergent behaviour.