Secrets of Edge-Informed Contrast Maximization for Event-Based Vision

TL;DR

This paper introduces a hybrid contrast maximization method that incorporates edge information to improve the sharpness and accuracy of event-based vision, achieving state-of-the-art results in optical flow estimation.

Contribution

It extends contrast maximization from uni-modal to bi-modal by integrating edge data, enhancing edge motion estimation accuracy.

Findings

Achieves superior sharpness scores in event images.

Sets new state-of-the-art optical flow benchmarks.

Demonstrates effectiveness on multiple datasets.

Abstract

Event cameras capture the motion of intensity gradients (edges) in the image plane in the form of rapid asynchronous events. When accumulated in 2D histograms, these events depict overlays of the edges in motion, consequently obscuring the spatial structure of the generating edges. Contrast maximization (CM) is an optimization framework that can reverse this effect and produce sharp spatial structures that resemble the moving intensity gradients by estimating the motion trajectories of the events. Nonetheless, CM is still an underexplored area of research with avenues for improvement. In this paper, we propose a novel hybrid approach that extends CM from uni-modal (events only) to bi-modal (events and edges). We leverage the underpinning concept that, given a reference time, optimally warped events produce sharp gradients consistent with the moving edge at that time. Specifically, we formalize a correlation-based objective to aid CM and provide key insights into the incorporation of multiscale and multireference techniques. Moreover, our edge-informed CM method yields superior sharpness scores and establishes new state-of-the-art event optical flow benchmarks on the MVSEC, DSEC, and ECD datasets.