Real-time Structure Flow

TL;DR

This paper introduces the structure flow field, a high-speed, dense 3D scene flow representation with an elegant PDE-based evolution model, enabling real-time motion prediction for dynamic robotics and autonomous vehicles.

Contribution

It presents a novel structure flow model with a PDE-based evolution and a real-time predictor-update algorithm for high-speed scene flow estimation.

Findings

Runs up to 600 Hz on GPU for 512x512 images

Validated on synthetic high-speed sequences

Demonstrated on real driving videos

Abstract

This article introduces the structure flow field; a flow field that can provide high-speed robo-centric motion information for motion control of highly dynamic robotic devices and autonomous vehicles. Structure flow is the angular 3D velocity of the scene at a given pixel. We show that structure flow posses an elegant evolution model in the form of a Partial Differential Equation (PDE) that enables us to create dense flow predictions forward in time. We exploit this structure to design a predictor-update algorithm to compute structure flow in real time using image and depth measurements. The prediction stage takes the previous estimate of the structure flow and propagates it forward in time using a numerical implementation of the structure flow PDE. The predicted flow is then updated using new image and depth data. The algorithm runs up to 600 Hz on a Desktop GPU machine for 512x512 images with flow values up to 8 pixels. We provide ground truth validation on high-speed synthetic image sequences as well as results on real-life video on driving scenarios.