DART: Distribution Aware Retinal Transform for Event-based Cameras

TL;DR

This paper introduces DART, a novel distribution-aware retinal transform descriptor for event cameras, applicable to object classification, tracking, detection, and feature matching, demonstrating robustness and versatility across multiple tasks.

Contribution

The paper presents DART, a new log-polar grid-based descriptor for event cameras, and applies it to multiple vision tasks with innovative methods for one-shot learning and long-term tracking.

Findings

DART achieves high accuracy in object classification on standard datasets.

The tracking method with scale and rotation equivariance improves robustness.

Object detection with cluster voting enhances long-term tracking performance.

Abstract

We introduce a generic visual descriptor, termed as distribution aware retinal transform (DART), that encodes the structural context using log-polar grids for event cameras. The DART descriptor is applied to four different problems, namely object classification, tracking, detection and feature matching: (1) The DART features are directly employed as local descriptors in a bag-of-features classification framework and testing is carried out on four standard event-based object datasets (N-MNIST, MNIST-DVS, CIFAR10-DVS, NCaltech-101). (2) Extending the classification system, tracking is demonstrated using two key novelties: (i) For overcoming the low-sample problem for the one-shot learning of a binary classifier, statistical bootstrapping is leveraged with online learning; (ii) To achieve tracker robustness, the scale and rotation equivariance property of the DART descriptors is exploited for the one-shot learning. (3) To solve the long-term object tracking problem, an object detector is designed using the principle of cluster majority voting. The detection scheme is then combined with the tracker to result in a high intersection-over-union score with augmented ground truth annotations on the publicly available event camera dataset. (4) Finally, the event context encoded by DART greatly simplifies the feature correspondence problem, especially for spatio-temporal slices far apart in time, which has not been explicitly tackled in the event-based vision domain.