Street-View Image Generation from a Bird's-Eye View Layout

TL;DR

This paper introduces BEVGen, a conditional generative model that creates realistic street-view images from a bird's-eye view layout, enhancing autonomous driving simulation and perception.

Contribution

BEVGen employs a novel cross-view transformation with spatial attention to generate spatially consistent street images from BEV layouts, advancing data-driven simulation methods.

Findings

Accurately renders road and lane lines.

Generates diverse traffic scenes with different weather and lighting.

Achieves high spatial consistency in synthesized images.

Abstract

Bird's-Eye View (BEV) Perception has received increasing attention in recent years as it provides a concise and unified spatial representation across views and benefits a diverse set of downstream driving applications. At the same time, data-driven simulation for autonomous driving has been a focal point of recent research but with few approaches that are both fully data-driven and controllable. Instead of using perception data from real-life scenarios, an ideal model for simulation would generate realistic street-view images that align with a given HD map and traffic layout, a task that is critical for visualizing complex traffic scenarios and developing robust perception models for autonomous driving. In this paper, we propose BEVGen, a conditional generative model that synthesizes a set of realistic and spatially consistent surrounding images that match the BEV layout of a traffic scenario. BEVGen incorporates a novel cross-view transformation with spatial attention design which learns the relationship between cameras and map views to ensure their consistency. We evaluate the proposed model on the challenging NuScenes and Argoverse 2 datasets. After training, BEVGen can accurately render road and lane lines, as well as generate traffic scenes with diverse different weather conditions and times of day.