Smart Black Box 2.0: Efficient High-bandwidth Driving Data Collection based on Video Anomalies

TL;DR

The paper introduces an enhanced Smart Black Box system for autonomous vehicles that uses anomaly detection to prioritize high-bandwidth video data collection, improving data relevance and storage efficiency.

Contribution

It extends the SBB system with real-time anomaly detection and evaluates its performance on a new real-world dataset, demonstrating improved data compression and prioritization.

Findings

Anomalous data compression increased by ~25%.

Prioritized recording improved anomalous-to-normal data ratio.

Real-world dataset evaluation validated the system's effectiveness.

Abstract

Autonomous vehicles require fleet-wide data collection for continuous algorithm development and validation. The Smart Black Box (SBB) intelligent event data recorder has been proposed as a system for prioritized high-bandwidth data capture. This paper extends the SBB by applying anomaly detection and action detection methods for generalized event-of-interest (EOI) detection. An updated SBB pipeline is proposed for the real-time capture of driving video data. A video dataset is constructed to evaluate the SBB on real-world data for the first time. SBB performance is assessed by comparing the compression of normal and anomalous data and by comparing our prioritized data recording with a FIFO strategy. Results show that SBB data compression can increase the anomalous-to-normal memory ratio by ~25%, while the prioritized recording strategy increases the anomalous-to-normal count ratio when compared to a FIFO strategy. We compare the real-world dataset SBB results to a baseline SBB given ground-truth anomaly labels and conclude that improved general EOI detection methods will greatly improve SBB performance.