Evaluating the Reliability of CNN Models on Classifying Traffic and Road Signs using LIME

TL;DR

This paper assesses the accuracy and interpretability of four pre-trained CNN models in classifying traffic signs using LIME, highlighting the importance of feature-based explanations for reliable model predictions.

Contribution

The study compares four CNN models' performance and employs LIME to enhance interpretability, providing insights into their reliability in traffic sign classification.

Findings

All models achieved high accuracy with F1 scores around 0.99.

LIME effectively explains model predictions and identifies relevant features.

Interpretability improves trust and dependability of traffic sign classifiers.

Abstract

The objective of this investigation is to evaluate and contrast the effectiveness of four state-of-the-art pre-trained models, ResNet-34, VGG-19, DenseNet-121, and Inception V3, in classifying traffic and road signs with the utilization of the GTSRB public dataset. The study focuses on evaluating the accuracy of these models' predictions as well as their ability to employ appropriate features for image categorization. To gain insights into the strengths and limitations of the model's predictions, the study employs the local interpretable model-agnostic explanations (LIME) framework. The findings of this experiment indicate that LIME is a crucial tool for improving the interpretability and dependability of machine learning models for image identification, regardless of the models achieving an f1 score of 0.99 on classifying traffic and road signs. The conclusion of this study has important ramifications for how these models are used in practice, as it is crucial to ensure that model predictions are founded on the pertinent image features.