Wayformer: Motion Forecasting via Simple & Efficient Attention Networks

TL;DR

Wayformer introduces a simple, homogeneous attention-based architecture for motion forecasting in autonomous driving, effectively integrating diverse input modalities and achieving state-of-the-art results with improved scalability and efficiency.

Contribution

The paper proposes a unified, attention-based model architecture that simplifies motion forecasting by using homogeneous components and explores different fusion strategies for input modalities.

Findings

Early fusion achieves state-of-the-art results on WOMD and Argoverse datasets.

Factorized and latent query attention strategies offer tradeoffs between efficiency and quality.

Wayformer outperforms complex, modality-specific systems in accuracy and scalability.

Abstract

Motion forecasting for autonomous driving is a challenging task because complex driving scenarios result in a heterogeneous mix of static and dynamic inputs. It is an open problem how best to represent and fuse information about road geometry, lane connectivity, time-varying traffic light state, and history of a dynamic set of agents and their interactions into an effective encoding. To model this diverse set of input features, many approaches proposed to design an equally complex system with a diverse set of modality specific modules. This results in systems that are difficult to scale, extend, or tune in rigorous ways to trade off quality and efficiency. In this paper, we present Wayformer, a family of attention based architectures for motion forecasting that are simple and homogeneous. Wayformer offers a compact model description consisting of an attention based scene encoder and a decoder. In the scene encoder we study the choice of early, late and hierarchical fusion of the input modalities. For each fusion type we explore strategies to tradeoff efficiency and quality via factorized attention or latent query attention. We show that early fusion, despite its simplicity of construction, is not only modality agnostic but also achieves state-of-the-art results on both Waymo Open MotionDataset (WOMD) and Argoverse leaderboards, demonstrating the effectiveness of our design philosophy