Recurrent Flow-Guided Semantic Forecasting

TL;DR

This paper introduces a real-time, efficient model for semantic forecasting in autonomous systems by decomposing the task into segmentation and optical flow prediction, achieving state-of-the-art accuracy with significantly reduced complexity.

Contribution

It proposes a novel decomposition approach and a lightweight model architecture that greatly improves efficiency and accuracy in semantic forecasting for autonomous systems.

Findings

Achieves state-of-the-art accuracy in short-term semantic forecasting.

Reduces model parameters by up to 95%.

Increases efficiency by over 40 times.

Abstract

Understanding the world around us and making decisions about the future is a critical component to human intelligence. As autonomous systems continue to develop, their ability to reason about the future will be the key to their success. Semantic anticipation is a relatively under-explored area for which autonomous vehicles could take advantage of (e.g., forecasting pedestrian trajectories). Motivated by the need for real-time prediction in autonomous systems, we propose to decompose the challenging semantic forecasting task into two subtasks: current frame segmentation and future optical flow prediction. Through this decomposition, we built an efficient, effective, low overhead model with three main components: flow prediction network, feature-flow aggregation LSTM, and end-to-end learnable warp layer. Our proposed method achieves state-of-the-art accuracy on short-term and moving objects semantic forecasting while simultaneously reducing model parameters by up to 95% and increasing efficiency by greater than 40x.