Latency-aware Road Anomaly Segmentation in Videos: A Photorealistic Dataset and New Metrics

TL;DR

This paper introduces the first video-based road anomaly segmentation dataset for autonomous driving, emphasizing temporal consistency and latency-aware metrics to improve safety-critical applications.

Contribution

It provides a synthetic, photorealistic dataset with new metrics for evaluating temporal and latency-aware segmentation in autonomous driving.

Findings

Benchmark baselines established for the new dataset

Introduction of temporal consistency and latency-aware metrics

Synthetic dataset with 120,000 high-res frames from multiple towns

Abstract

In the past several years, road anomaly segmentation is actively explored in the academia and drawing growing attention in the industry. The rationale behind is straightforward: if the autonomous car can brake before hitting an anomalous object, safety is promoted. However, this rationale naturally calls for a temporally informed setting while existing methods and benchmarks are designed in an unrealistic frame-wise manner. To bridge this gap, we contribute the first video anomaly segmentation dataset for autonomous driving. Since placing various anomalous objects on busy roads and annotating them in every frame are dangerous and expensive, we resort to synthetic data. To improve the relevance of this synthetic dataset to real-world applications, we train a generative adversarial network conditioned on rendering G-buffers for photorealism enhancement. Our dataset consists of 120,000 high-resolution frames at a 60 FPS framerate, as recorded in 7 different towns. As an initial benchmarking, we provide baselines using latest supervised and unsupervised road anomaly segmentation methods. Apart from conventional ones, we focus on two new metrics: temporal consistency and latencyaware streaming accuracy. We believe the latter is valuable as it measures whether an anomaly segmentation algorithm can truly prevent a car from crashing in a temporally informed setting.