Asynchronous Multi-Object Tracking with an Event Camera

TL;DR

This paper introduces AEMOT, an asynchronous multi-object tracking algorithm for event cameras, which detects and tracks multiple objects with high accuracy in dynamic environments, outperforming existing methods.

Contribution

The paper presents a novel asynchronous tracking algorithm that processes raw events, uses a new feature detection method, and introduces a validation stage, advancing event-based multi-object tracking.

Findings

AEMOT outperforms existing algorithms by over 37% in accuracy.

It effectively tracks small objects like bees in real-time.

The approach is validated on a new Bee Swarm Dataset.

Abstract

Events cameras are ideal sensors for enabling robots to detect and track objects in highly dynamic environments due to their low latency output, high temporal resolution, and high dynamic range. In this paper, we present the Asynchronous Event Multi-Object Tracking (AEMOT) algorithm for detecting and tracking multiple objects by processing individual raw events asynchronously. AEMOT detects salient event blob features by identifying regions of consistent optical flow using a novel Field of Active Flow Directions built from the Surface of Active Events. Detected features are tracked as candidate objects using the recently proposed Asynchronous Event Blob (AEB) tracker in order to construct small intensity patches of each candidate object. A novel learnt validation stage promotes or discards candidate objects based on classification of their intensity patches, with promoted objects having their position, velocity, size, and orientation estimated at their event rate. We evaluate AEMOT on a new Bee Swarm Dataset, where it tracks dozens of small bees with precision and recall performance exceeding that of alternative event-based detection and tracking algorithms by over 37%. Source code and the labelled event Bee Swarm Dataset will be open sourced