CCLSTM: Coupled Convolutional Long-Short Term Memory Network for Occupancy Flow Forecasting

TL;DR

This paper introduces CCLSTM, a lightweight convolutional LSTM model that effectively predicts occupancy flow in autonomous driving, achieving state-of-the-art results without relying on complex inputs or transformer architectures.

Contribution

The paper presents CCLSTM, a novel convolutional LSTM architecture that captures spatial-temporal dynamics for occupancy flow prediction without vectorized inputs or self-attention.

Findings

Achieves state-of-the-art performance on occupancy flow metrics.

Ranks first in all metrics on the 2024 Waymo Challenge leaderboard.

Operates efficiently as a lightweight, end-to-end trainable model.

Abstract

Predicting future states of dynamic agents is a fundamental task in autonomous driving. An expressive representation for this purpose is Occupancy Flow Fields, which provide a scalable and unified format for modeling motion, spatial extent, and multi-modal future distributions. While recent methods have achieved strong results using this representation, they often depend on high-quality vectorized inputs, which are unavailable or difficult to generate in practice, and the use of transformer-based architectures, which are computationally intensive and costly to deploy. To address these issues, we propose \textbf{Coupled Convolutional LSTM (CCLSTM)}, a lightweight, end-to-end trainable architecture based solely on convolutional operations. Without relying on vectorized inputs or self-attention mechanisms, CCLSTM effectively captures temporal dynamics and spatial occupancy-flow correlations using a compact recurrent convolutional structure. Despite its simplicity, CCLSTM achieves state-of-the-art performance on occupancy flow metrics and, as of this submission, ranks \(1^{\text{st}}\) in all metrics on the 2024 Waymo Occupancy and Flow Prediction Challenge leaderboard.