Looped SSMs: Depth-Recurrence and Input Reshaping for Time Series Classification

TL;DR

This paper introduces depth-recurrence and input reshaping techniques for State Space Models, demonstrating their effectiveness in improving time series classification accuracy and efficiency.

Contribution

It explores depth-recurrence in SSMs, showing parameter sharing across layers enhances performance, and highlights input reshaping as a key design axis for better results.

Findings

Looped SSMs match or outperform standard SSMs with fewer parameters.

Depth-recurrence provides benefits beyond expressivity, aiding optimization.

Input reshaping improves accuracy by 1-6% across models.

Abstract

State Space Models (SSMs) are inherently recurrent along the sequence dimension, yet depth-recurrence - reusing the same block repeatedly across layers, as recently applied in looped transformers - has not been explored in this model family. We show that a looped SSM with $k$ parameters iterated $L$ times consistently closely matches or outperforms a standard SSM with $k \cdot L$ independent parameters across four architectures (LRU, S5, LinOSS, LrcSSM) and six time series classification benchmarks, despite operating within a strictly smaller hypothesis space, as we formally establish. Since the larger model contains the looped model as a special case, this dominance cannot be explained by expressivity and instead points to parameter sharing across depth as a beneficial inductive bias that simplifies optimization. These results demonstrate that depth-recurrence is orthogonal to sequence-recurrence and independently beneficial. We further show that input reshaping is an equally neglected design axis: concatenating timesteps for low-dimensional inputs, or flattening and rechunking the joint feature-time dimension for high-dimensional ones, yields accuracy gains of 1-6% across all models, confirmed over 5 random seeds. Both techniques provide standalone improvements that compound when combined, suggesting that depth and input reshaping are two independent and underexplored design axes for SSMs on time series.