Learning Normal Flow Directly From Event Neighborhoods

TL;DR

This paper introduces a novel supervised point-based normal flow estimation method from event data, improving transferability, robustness, and uncertainty quantification over existing approaches, and demonstrates superior performance across datasets.

Contribution

A new point-based normal flow estimator from raw events that enhances transferability, robustness, and uncertainty quantification compared to prior model-based and learning-based methods.

Findings

Outperforms state-of-the-art methods in cross-dataset transferability

Supports diverse data augmentation techniques for robustness

Enables effective egomotion estimation using normal flow and IMU data

Abstract

Event-based motion field estimation is an important task. However, current optical flow methods face challenges: learning-based approaches, often frame-based and relying on CNNs, lack cross-domain transferability, while model-based methods, though more robust, are less accurate. To address the limitations of optical flow estimation, recent works have focused on normal flow, which can be more reliably measured in regions with limited texture or strong edges. However, existing normal flow estimators are predominantly model-based and suffer from high errors. In this paper, we propose a novel supervised point-based method for normal flow estimation that overcomes the limitations of existing event learning-based approaches. Using a local point cloud encoder, our method directly estimates per-event normal flow from raw events, offering multiple unique advantages: 1) It produces temporally and spatially sharp predictions. 2) It supports more diverse data augmentation, such as random rotation, to improve robustness across various domains. 3) It naturally supports uncertainty quantification via ensemble inference, which benefits downstream tasks. 4) It enables training and inference on undistorted data in normalized camera coordinates, improving transferability across cameras. Extensive experiments demonstrate our method achieves better and more consistent performance than state-of-the-art methods when transferred across different datasets. Leveraging this transferability, we train our model on the union of datasets and release it for public use. Finally, we introduce an egomotion solver based on a maximum-margin problem that uses normal flow and IMU to achieve strong performance in challenging scenarios.