Brain-inspired spike-timing plasticity for reliable label-efficient event-camera vision

TL;DR

This paper introduces spike-timing-dependent plasticity modules for event-camera vision that operate efficiently on CPU, achieving high accuracy with minimal supervision and demonstrating robustness under data drift.

Contribution

It presents novel local STDP modules enabling label-efficient, GPU-free event-camera object detection with improved robustness and transferability.

Findings

Zero-label detector achieves 53.8% mAP@30.

STDP modules reach up to 78.60% mAP@30 on FRED benchmark.

Tube-level STDP reduces false alarms significantly.

Abstract

Deploying event-camera object detectors is constrained by per-frame labeling requirements and GPU compute demands. This work introduces three local spike-timing-dependent plasticity (STDP) modules, including sequence, candidate, and tube-reliability modules, that operate on a single CPU thread without GPU support. On the FRED drone benchmark, the proposed framework spans three label-efficient supervision tiers. A strict zero-label detector achieves 53.8% mAP@30, approximately 26 train-derived bits achieve 76.9% mAP@30, and an STDP candidate-reliability gate achieves 78.60 +/- 0.42% mAP@30. Under acquisition-order drift, the cohort gate outperforms streaming k-means by 2.03 +/- 0.58 percentage points across 20 of 20 positive trials, while a no-drift control falsifies the effect. STDP reduces single-model variance by 6.6 times, and one trained gate matches a 44-seed ensemble bound. The gate transfers to Intel Lava with 89% top-2 agreement. On the EVUAV benchmark, a tube-level STDP layer reduces false alarms from 454 to 331e-4 at Pd >= 88%. Dense gradient-trained detectors cannot provide this combination of gradient training, dense matrix multiplication, and local plasticity-free operation by construction.