BiDepth: A Bidirectional-Depth Neural Network for Spatio-Temporal Prediction

TL;DR

BiDepth introduces a bidirectional-depth neural network with a novel self-attention mechanism that effectively captures complex spatio-temporal correlations, improving prediction accuracy in urban mobility and weather data.

Contribution

The paper presents the BiDepth neural network with dynamic depth modulation and a convolutional self-attention cell, enhancing spatio-temporal modeling capabilities over existing methods.

Findings

12% MSE reduction in urban traffic prediction

15% improvement in precipitation forecasting

Superior performance over ConvLSTM on real-world datasets

Abstract

Accurate spatial-temporal (ST) prediction for dynamic systems, such as urban mobility and weather patterns, is crucial but hindered by complex ST correlations and the challenge of concurrently modeling long-term trends with short-term fluctuations. Existing methods often falter in these areas. This paper proposes the BiDepth Multimodal Neural Network (BDMNN), which integrates two key innovations: 1) a bidirectional depth modulation mechanism that dynamically adjusts network depth to comprehensively capture both long-term seasonality and immediate short-term events; and 2) a novel convolutional self-attention cell (CSAC). Critically, unlike many attention mechanisms that can lose spatial acuity, our CSAC is specifically designed to preserve crucial spatial relationships throughout the network, akin to standard convolutional layers, while simultaneously capturing temporal dependencies. Evaluated on real-world urban traffic and precipitation datasets, BDMNN demonstrates significant accuracy improvements, achieving a 12% Mean Squared Error (MSE) reduction in urban traffic prediction and a 15% improvement in precipitation forecasting over leading deep learning benchmarks like ConvLSTM, using comparable computational resources. These advancements offer robust ST forecasting for smart city management, disaster prevention, and resource optimization.