A-Eye: Driving with the Eyes of AI for Corner Case Generation

TL;DR

This paper introduces A-Eye, a system that generates synthetic corner cases for autonomous driving by using a human-in-the-loop approach with a semantic segmentation network integrated into CARLA, improving pedestrian detection.

Contribution

The paper presents a novel test rig that creates corner cases for autonomous driving by combining real-time semantic segmentation with human intervention in simulation.

Findings

Enrichment of training data with corner cases improves pedestrian detection.

The system effectively identifies critical scenarios that challenge perception algorithms.

Human-in-the-loop approach enhances the generation of meaningful corner cases.

Abstract

The overall goal of this work is to enrich training data for automated driving with so called corner cases. In road traffic, corner cases are critical, rare and unusual situations that challenge the perception by AI algorithms. For this purpose, we present the design of a test rig to generate synthetic corner cases using a human-in-the-loop approach. For the test rig, a real-time semantic segmentation network is trained and integrated into the driving simulation software CARLA in such a way that a human can drive on the network's prediction. In addition, a second person gets to see the same scene from the original CARLA output and is supposed to intervene with the help of a second control unit as soon as the semantic driver shows dangerous driving behavior. Interventions potentially indicate poor recognition of a critical scene by the segmentation network and then represents a corner case. In our experiments, we show that targeted enrichment of training data with corner cases leads to improvements in pedestrian detection in safety relevant episodes in road traffic.