Single Image Optical Flow Estimation with an Event Camera

TL;DR

This paper introduces a novel optical flow estimation method that leverages event camera data and handles both blurred and sharp images, improving accuracy over existing techniques.

Contribution

It presents a unified model that uses event data and accounts for image blur, enhancing optical flow estimation from a single image and event stream.

Findings

Outperforms state-of-the-art methods on synthetic and real datasets.

Effectively handles blurred and sharp images for flow estimation.

Demonstrates improved accuracy using event-based photometric consistency.

Abstract

Event cameras are bio-inspired sensors that asynchronously report intensity changes in microsecond resolution. DAVIS can capture high dynamics of a scene and simultaneously output high temporal resolution events and low frame-rate intensity images. In this paper, we propose a single image (potentially blurred) and events based optical flow estimation approach. First, we demonstrate how events can be used to improve flow estimates. To this end, we encode the relation between flow and events effectively by presenting an event-based photometric consistency formulation. Then, we consider the special case of image blur caused by high dynamics in the visual environments and show that including the blur formation in our model further constrains flow estimation. This is in sharp contrast to existing works that ignore the blurred images while our formulation can naturally handle either blurred or sharp images to achieve accurate flow estimation. Finally, we reduce flow estimation, as well as image deblurring, to an alternative optimization problem of an objective function using the primal-dual algorithm. Experimental results on both synthetic and real data (with blurred and non-blurred images) show the superiority of our model in comparison to state-of-the-art approaches.