Forecasting Future Instance Segmentation with Learned Optical Flow and Warping

TL;DR

This paper introduces a model that uses learned optical flow and warping to predict future instance segmentations for autonomous vehicles, enhancing scene understanding and safety predictions.

Contribution

It proposes an autoregressive flow forecasting model combined with learned warping to improve future segmentation predictions in autonomous driving scenarios.

Findings

Optical flow-based predictions outperform baseline methods.

The model effectively forecasts future scene segmentations.

Results demonstrate improved accuracy on Cityscapes dataset.

Abstract

For an autonomous vehicle it is essential to observe the ongoing dynamics of a scene and consequently predict imminent future scenarios to ensure safety to itself and others. This can be done using different sensors and modalities. In this paper we investigate the usage of optical flow for predicting future semantic segmentations. To do so we propose a model that forecasts flow fields autoregressively. Such predictions are then used to guide the inference of a learned warping function that moves instance segmentations on to future frames. Results on the Cityscapes dataset demonstrate the effectiveness of optical-flow methods.