A Set-Sequence Model for Time Series

TL;DR

Set-Sequence is a novel model that captures cross-sectional effects in large collections of time series without manual feature engineering, improving prediction accuracy and interpretability in finance and economics applications.

Contribution

It introduces a permutation-invariant Set module integrated with sequence models, enabling direct learning of cross-sectional structure and scalable prediction across unaligned and varying-sized series.

Findings

Outperforms baselines in synthetic contagion task

Achieves higher Sharpe ratios in portfolio optimization

Improves AUC in loan risk prediction

Abstract

Many prediction problems across science and engineering, especially in finance and economics, involve large cross-sections of individual time series, where each unit (e.g., a loan, stock, or customer) is driven by unit-level features and latent cross-sectional dynamics. While sequence models have advanced per-unit temporal prediction, capturing cross-sectional effects often still relies on hand-crafted summary features. We propose Set-Sequence, a model that learns cross-sectional structure directly, enhancing expressivity and eliminating manual feature engineering. At each time step, a permutation-invariant Set module summarizes the unit set; a Sequence module then models each unit's dynamics conditioned on both its features and the learned summary. The architecture accommodates unaligned series, supports varying numbers of units at inference, integrates with standard sequence backbones (e.g., Transformers), and scales linearly in cross-sectional size. Across a synthetic contagion task and two large-scale real-world applications, equity portfolio optimization and loan risk prediction, Set-Sequence significantly outperforms strong baselines, delivering higher Sharpe ratios, improved AUCs, and interpretable cross-sectional summaries.