Unbiased Gradient Estimation for Event Binning via Functional Backpropagation

TL;DR

This paper introduces a novel unbiased gradient estimation method for event binning functions in event-based vision, enabling more efficient learning from raw events and improving performance in tasks like egomotion, optical flow, and SLAM.

Contribution

The authors propose a new framework for unbiased gradient estimation of binning functions using weak derivatives via integration by parts, enhancing learning from event data.

Findings

Improved egomotion estimation with 3.2% lower RMS error

Achieved 9.4% lower EPE in self-supervised optical flow

Reduced RMS error in SLAM by 5.1%

Abstract

Event-based vision encodes dynamic scenes as asynchronous spatio-temporal spikes called events. To leverage conventional image processing pipelines, events are typically binned into frames. However, binning functions are discontinuous, which truncates gradients at the frame level and forces most event-based algorithms to rely solely on frame-based features. Attempts to directly learn from raw events avoid this restriction but instead suffer from biased gradient estimation due to the discontinuities of the binning operation, ultimately limiting their learning efficiency. To address this challenge, we propose a novel framework for unbiased gradient estimation of arbitrary binning functions by synthesizing weak derivatives during backpropagation while keeping the forward output unchanged. The key idea is to exploit integration by parts: lifting the target functions to functionals yields an integral form of the derivative of the binning function during backpropagation, where the cotangent function naturally arises. By reconstructing this cotangent function from the sampled cotangent vector, we compute weak derivatives that provably match long-range finite differences of both smooth and non-smooth targets. Experimentally, our method improves simple optimization-based egomotion estimation with 3.2\% lower RMS error and 1.57$\times$ faster convergence. On complex downstream tasks, we achieve 9.4\% lower EPE in self-supervised optical flow, and 5.1\% lower RMS error in SLAM, demonstrating broad benefits for event-based visual perception. Source code can be found at https://github.com/chjz1024/EventFBP.