Filter-Based Reconstruction of Images from Events

TL;DR

FIBAR is a simple, asynchronous, filter-based method for reconstructing images from event camera data, operating efficiently on CPUs and suitable for tasks like fiducial marker detection, offering an alternative to neural network approaches.

Contribution

The paper introduces FIBAR, a novel filter-based asynchronous reconstruction method that simplifies image reconstruction from event data without neural networks.

Findings

Runs on CPU at 42-140 million events/sec

Produces noisier images than neural networks but effective for fiducial detection

Operates asynchronously allowing flexible image read-out times

Abstract

Reconstructing an intensity image from the events of a moving event camera is a challenging task that is typically approached with neural networks deployed on graphics processing units. This paper presents a much simpler, FIlter Based Asynchronous Reconstruction method (FIBAR). First, intensity changes signaled by events are integrated with a temporal digital IIR filter. To reduce reconstruction noise, stale pixels are detected by a novel algorithm that regulates a window of recently updated pixels. Arguing that for a moving camera, the absence of events at a pixel location likely implies a low image gradient, stale pixels are then blurred with a Gaussian filter. In contrast to most existing methods, FIBAR is asynchronous and permits image read-out at an arbitrary time. It runs on a modern laptop CPU at about 42(140) million events/s with (without) spatial filtering enabled. A few simple qualitative experiments are presented that show the difference in image reconstruction between FIBAR and a neural network-based approach (FireNet). FIBAR's reconstruction is noisier than neural network-based methods and suffers from ghost images. However, it is sufficient for certain tasks such as the detection of fiducial markers. Code is available at https://github.com/ros-event-camera/event_image_reconstruction_fibar