Automated Quality Check of Sensor Data Annotations

TL;DR

This paper introduces an open-source tool for automatically detecting errors in multi-sensor railway datasets, enhancing data quality assurance for training AI in automated driving systems.

Contribution

The paper presents a novel automatic quality check method and an open-source tool for detecting common errors in multi-sensor data for railway vehicles.

Findings

Six detection methods achieved 100% precision.

Three additional methods reached 96-97% precision.

Significantly reduces manual effort in data quality assurance.

Abstract

The monitoring of the route and track environment plays an important role in automated driving. For example, it can be used as an assistance system for route monitoring in automation level Grade of Automation (GoA) 2, where the train driver is still on board. In fully automated, driverless driving at automation level GoA4, these systems finally take over environment monitoring completely independently. With the help of artificial intelligence (AI), they react automatically to risks and dangerous events on the route. To train such AI algorithms, large amounts of training data are required, which must meet high-quality standards due to their safety relevance. In this publication we present an automatic method for assuring the quality of training data, significantly reducing the manual workload and accelerating the development of these systems. We propose an open-source tool designed to detect nine common errors found in multi-sensor datasets for railway vehicles. To evaluate the performance of the framework, all detected errors were manually validated. Six issue detection methods achieved 100% precision, while three additional methods reached precision rates 96% and 97%.