SignRAG: A Retrieval-Augmented System for Scalable Zero-Shot Road Sign Recognition
Minghao Zhu, Zhihao Zhang, Anmol Sidhu, Keith Redmill

TL;DR
This paper presents SignRAG, a zero-shot road sign recognition system that combines vision-language and language models with retrieval techniques to accurately identify signs without extensive labeled datasets.
Contribution
It introduces a novel retrieval-augmented recognition framework that leverages VLMs and LLMs for scalable zero-shot road sign identification, bypassing the need for exhaustive training.
Findings
Achieves 95.58% accuracy on ideal images
Attains 82.45% accuracy on real-world data
Demonstrates effectiveness of RAG architecture for scalable recognition
Abstract
Automated road sign recognition is a critical task for intelligent transportation systems, but traditional deep learning methods struggle with the sheer number of sign classes and the impracticality of creating exhaustive labeled datasets. This paper introduces a novel zero-shot recognition framework that adapts the Retrieval-Augmented Generation (RAG) paradigm to address this challenge. Our method first uses a Vision Language Model (VLM) to generate a textual description of a sign from an input image. This description is used to retrieve a small set of the most relevant sign candidates from a vector database of reference designs. Subsequently, a Large Language Model (LLM) reasons over the retrieved candidates to make a final, fine-grained recognition. We validate this approach on a comprehensive set of 303 regulatory signs from the Ohio MUTCD. Experimental results demonstrate the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Neural Network Applications · Hand Gesture Recognition Systems · Autonomous Vehicle Technology and Safety
