TSformer: A Non-autoregressive Spatial-temporal Transformers for 30-day Ocean Eddy-Resolving Forecasting

TL;DR

TSformer is a novel non-autoregressive spatiotemporal transformer model that achieves 30-day ocean eddy-resolving forecasts with high accuracy and computational efficiency, maintaining physical consistency over extended periods.

Contribution

This paper introduces TSformer, a hierarchical U-Net transformer architecture that extends local attention to spatiotemporal contexts, enabling faster and more stable medium-range ocean forecasting.

Findings

Comparable performance to leading numerical models

Orders of magnitude faster computation

Maintains 3D physical consistency in long-term forecasts

Abstract

Ocean forecasting is critical for various applications and is essential for understanding air-sea interactions, which contribute to mitigating the impacts of extreme events. State-of-the-art ocean numerical forecasting systems can offer lead times of up to 10 days with a spatial resolution of 10 kilometers, although they are computationally expensive. While data-driven forecasting models have demonstrated considerable potential and speed, they often primarily focus on spatial variations while neglecting temporal dynamics. This paper presents TSformer, a novel non-autoregressive spatiotemporal transformer designed for medium-range ocean eddy-resolving forecasting, enabling forecasts of up to 30 days in advance. We introduce an innovative hierarchical U-Net encoder-decoder architecture based on 3D Swin Transformer blocks, which extends the scope of local attention computation from spatial to spatiotemporal contexts to reduce accumulation errors. TSformer is trained on 28 years of homogeneous, high-dimensional 3D ocean reanalysis datasets, supplemented by three 2D remote sensing datasets for surface forcing. Based on the near-real-time operational forecast results from 2023, comparative performance assessments against in situ profiles and satellite observation data indicate that, TSformer exhibits forecast performance comparable to leading numerical ocean forecasting models while being orders of magnitude faster. Unlike autoregressive models, TSformer maintains 3D consistency in physical motion, ensuring long-term coherence and stability in extended forecasts. Furthermore, the TSformer model, which incorporates surface auxiliary observational data, effectively simulates the vertical cooling and mixing effects induced by Super Typhoon Saola.