TopoFormer: Integrating Transformers and ConvLSTMs for Coastal Topography Prediction

TL;DR

TopoFormer is a hybrid deep learning model combining transformers and ConvLSTMs, designed for accurate coastal topography prediction, effectively capturing temporal and spatial patterns in beach profile data.

Contribution

The paper introduces TopoFormer, a novel architecture integrating transformers and ConvLSTMs for improved topographic beach profile prediction.

Findings

Achieved as low as 2 cm MAE in predictions.

Outperformed state-of-the-art models in accuracy.

Proved effective in both in-distribution and out-of-distribution scenarios.

Abstract

This paper presents \textit{TopoFormer}, a novel hybrid deep learning architecture that integrates transformer-based encoders with convolutional long short-term memory (ConvLSTM) layers for the precise prediction of topographic beach profiles referenced to elevation datums, with a particular focus on Mean Low Water Springs (MLWS) and Mean Low Water Neaps (MLWN). Accurate topographic estimation down to MLWS is critical for coastal management, navigation safety, and environmental monitoring. Leveraging a comprehensive dataset from the Wales Coastal Monitoring Centre (WCMC), consisting of over 2000 surveys across 36 coastal survey units, TopoFormer addresses key challenges in topographic prediction, including temporal variability and data gaps in survey measurements. The architecture uniquely combines multi-head attention mechanisms and ConvLSTM layers to capture both long-range dependencies and localized temporal patterns inherent in beach profiles data. TopoFormer's predictive performance was rigorously evaluated against state-of-the-art models, including DenseNet, 1D/2D CNNs, and LSTMs. While all models demonstrated strong performance, \textit{TopoFormer} achieved the lowest mean absolute error (MAE), as low as 2 cm, and provided superior accuracy in both in-distribution (ID) and out-of-distribution (OOD) evaluations.