SignEye: Traffic Sign Interpretation from Vehicle First-Person View

TL;DR

This paper introduces SignEye, a novel approach for interpreting traffic signs from a vehicle's first-person view to enhance autonomous driving systems with spatial and semantic understanding, supporting navigation and regulation assessment.

Contribution

The paper proposes a new task TSI-FPV, develops the SignEye pipeline, and creates the Traffic-CN dataset, advancing traffic sign understanding in autonomous driving.

Findings

SignEye achieves effective traffic sign interpretation from first-person view.

TGA provides valuable supplementary information for autonomous driving.

Experiments validate the feasibility of TSI-FPV and TGA with the Traffic-CN dataset.

Abstract

Traffic signs play a key role in assisting autonomous driving systems (ADS) by enabling the assessment of vehicle behavior in compliance with traffic regulations and providing navigation instructions. However, current works are limited to basic sign understanding without considering the egocentric vehicle's spatial position, which fails to support further regulation assessment and direction navigation. Following the above issues, we introduce a new task: traffic sign interpretation from the vehicle's first-person view, referred to as TSI-FPV. Meanwhile, we develop a traffic guidance assistant (TGA) scenario application to re-explore the role of traffic signs in ADS as a complement to popular autonomous technologies (such as obstacle perception). Notably, TGA is not a replacement for electronic map navigation; rather, TGA can be an automatic tool for updating it and complementing it in situations such as offline conditions or temporary sign adjustments. Lastly, a spatial and semantic logic-aware stepwise reasoning pipeline (SignEye) is constructed to achieve the TSI-FPV and TGA, and an application-specific dataset (Traffic-CN) is built. Experiments show that TSI-FPV and TGA are achievable via our SignEye trained on Traffic-CN. The results also demonstrate that the TGA can provide complementary information to ADS beyond existing popular autonomous technologies.