LightFormer: An End-to-End Model for Intersection Right-of-Way Recognition Using Traffic Light Signals and an Attention Mechanism

TL;DR

LightFormer is an end-to-end deep learning model that uses attention mechanisms and temporal features to accurately recognize right-of-way status at signalized intersections, aiding autonomous vehicle decision-making.

Contribution

The paper introduces LightFormer, a novel model combining spatial-temporal attention and a modified loss function for improved intersection right-of-way recognition.

Findings

Achieved high accuracy on public traffic light datasets.

Effectively incorporates temporal information for better classification.

Demonstrated robustness in complex urban intersection scenarios.

Abstract

For smart vehicles driving through signalised intersections, it is crucial to determine whether the vehicle has right of way given the state of the traffic lights. To address this issue, camera based sensors can be used to determine whether the vehicle has permission to proceed straight, turn left or turn right. This paper proposes a novel end to end intersection right of way recognition model called LightFormer to generate right of way status for available driving directions in complex urban intersections. The model includes a spatial temporal inner structure with an attention mechanism, which incorporates features from past image to contribute to the classification of the current frame right of way status. In addition, a modified, multi weight arcface loss is introduced to enhance the model classification performance. Finally, the proposed LightFormer is trained and tested on two public traffic light datasets with manually augmented labels to demonstrate its effectiveness.