Neuromorphic Optical Flow and Real-time Implementation with Event Cameras

TL;DR

This paper introduces a novel, efficient neural network architecture for optical flow estimation using event cameras, achieving high accuracy and real-time performance with significantly reduced complexity, suitable for edge and robotic applications.

Contribution

The paper presents a new network inspired by Timelens that improves optical flow accuracy and a methodology for model simplification enabling real-time deployment with event cameras.

Findings

Achieves near two orders of magnitude reduction in complexity.

Maintains high accuracy in optical flow estimation.

Enables real-time processing with event cameras.

Abstract

Optical flow provides information on relative motion that is an important component in many computer vision pipelines. Neural networks provide high accuracy optical flow, yet their complexity is often prohibitive for application at the edge or in robots, where efficiency and latency play crucial role. To address this challenge, we build on the latest developments in event-based vision and spiking neural networks. We propose a new network architecture, inspired by Timelens, that improves the state-of-the-art self-supervised optical flow accuracy when operated both in spiking and non-spiking mode. To implement a real-time pipeline with a physical event camera, we propose a methodology for principled model simplification based on activity and latency analysis. We demonstrate high speed optical flow prediction with almost two orders of magnitude reduced complexity while maintaining the accuracy, opening the path for real-time deployments.