LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving

TL;DR

LookOut is a self-driving system that predicts diverse future scenarios and plans safe, adaptable trajectories, improving safety and efficiency in complex traffic environments.

Contribution

The paper introduces a novel generative approach for diverse multi-agent future prediction combined with contingency planning for self-driving cars.

Findings

More diverse and accurate motion forecasting

Safer and less conservative planning in simulations

Improved sample efficiency in predicting future trajectories

Abstract

In this paper, we present LookOut, a novel autonomy system that perceives the environment, predicts a diverse set of futures of how the scene might unroll and estimates the trajectory of the SDV by optimizing a set of contingency plans over these future realizations. In particular, we learn a diverse joint distribution over multi-agent future trajectories in a traffic scene that covers a wide range of future modes with high sample efficiency while leveraging the expressive power of generative models. Unlike previous work in diverse motion forecasting, our diversity objective explicitly rewards sampling future scenarios that require distinct reactions from the self-driving vehicle for improved safety. Our contingency planner then finds comfortable and non-conservative trajectories that ensure safe reactions to a wide range of future scenarios. Through extensive evaluations, we show that our model demonstrates significantly more diverse and sample-efficient motion forecasting in a large-scale self-driving dataset as well as safer and less-conservative motion plans in long-term closed-loop simulations when compared to current state-of-the-art models.