Accelerating Road Sign Ground Truth Construction with Knowledge Graph and Machine Learning

TL;DR

This paper introduces a novel system combining knowledge graphs and a variational prototyping-encoder to assist in road sign annotation, significantly reducing search effort and improving accuracy in diverse international contexts.

Contribution

The paper presents a new approach integrating knowledge graphs with machine learning to enhance road sign annotation efficiency and accuracy, addressing challenges faced by human annotators.

Findings

Reduces sign search space by 98.9%

Proposes correct candidate for 75% of signs

Eliminates human search effort in many cases

Abstract

Having a comprehensive, high-quality dataset of road sign annotation is critical to the success of AI-based Road Sign Recognition (RSR) systems. In practice, annotators often face difficulties in learning road sign systems of different countries; hence, the tasks are often time-consuming and produce poor results. We propose a novel approach using knowledge graphs and a machine learning algorithm - variational prototyping-encoder (VPE) - to assist human annotators in classifying road signs effectively. Annotators can query the Road Sign Knowledge Graph using visual attributes and receive closest matching candidates suggested by the VPE model. The VPE model uses the candidates from the knowledge graph and a real sign image patch as inputs. We show that our knowledge graph approach can reduce sign search space by 98.9%. Furthermore, with VPE, our system can propose the correct single candidate for 75% of signs in the tested datasets, eliminating the human search effort entirely in those cases.