Transferring dense object detection models to event-based data

TL;DR

This paper evaluates adapting the YOLO object detection model to event-based data by replacing dense convolutions with sparse or asynchronous sparse convolutions, analyzing performance and runtime implications.

Contribution

It introduces the application of sparse and asynchronous sparse convolutions to event-based data detection within YOLO, highlighting implementation challenges.

Findings

Sparse convolutions do not improve runtime despite lower theoretical computation.

Performance varies depending on convolution type used.

Direct processing of event data with sparse convolutions is feasible but not yet optimal.

Abstract

Event-based image representations are fundamentally different to traditional dense images. This poses a challenge to apply current state-of-the-art models for object detection as they are designed for dense images. In this work we evaluate the YOLO object detection model on event data. To this end we replace dense-convolution layers by either sparse convolutions or asynchronous sparse convolutions which enables direct processing of event-based images and compare the performance and runtime to feeding event-histograms into dense-convolutions. Here, hyper-parameters are shared across all variants to isolate the effect sparse-representation has on detection performance. At this, we show that current sparse-convolution implementations cannot translate their theoretical lower computation requirements into an improved runtime.