TIDES: Implicit Time-Awareness in Selective State Space Models

TL;DR

TIDES introduces a novel selective state space model that maintains physical time interpretation while enhancing expressivity, enabling better handling of irregular time series and achieving state-of-the-art results.

Contribution

It proposes moving input dependence from step size to the diagonal state matrix, reconciling selective and continuous SSM architectures.

Findings

TIDES handles irregular timestamps natively without losing expressivity.

It outperforms existing models on UEA time-series classification and Physiome-ODE regression benchmarks.

The model avoids failure modes of current architectures on a new diagnostic benchmark.

Abstract

Selective state space models (SSMs), such as Mamba, achieve strong per-token expressivity by making the time discretization step $\Tilde{\Delta}$ a learned function of the input. However, in doing so, $\Tilde{\Delta}$ ceases to represent a physical sampling interval, limiting its irregular time series modeling capability. Continuous-time SSMs, such as S5, preserve the physical meaning of $\Tilde{\Delta}$ and handle irregular timestamps natively ($\Tilde{\Delta}\equiv\Delta)$, but their dynamics remain linear time-invariant (LTI), limiting per-token expressivity. We propose \textbf{TIDES}, a selective SSM variant that reconciles selective and continuous architectures by moving input-dependence off the step size and onto the diagonal state matrix. As a result, $\Tilde{\Delta}$ retains its physical meaning, tied to the state discretization, allowing the model to handle irregular timestamps natively without sacrificing the per-token expressivity that makes selective SSMs effective. We show this on a novel \emph{Fading Flash} experimental benchmark, a compact controlled diagnostic for sequence models that jointly tests input-dependence and extrapolation to out-of-distribution $\Delta$ values, and isolates the distinct failure modes of current state-of-the-art architectures that TIDES avoids by construction. On large-scale benchmarks, TIDES sets the new state-of-the-art average rank on UEA time-series classification and the Physiome-ODE regression benchmark. Code available at: https://github.com/TaylanSoydan/TIDES.