A Comparative Performance Analysis of Classification and Segmentation Models on Bangladeshi Pothole Dataset

TL;DR

This study evaluates classification and segmentation models on a new Bangladeshi pothole dataset, demonstrating high accuracy and efficiency, with lightweight models performing comparably to heavier ones, and data augmentation improving results.

Contribution

Introduces a new, extensive pothole dataset from Bangladesh and provides a comprehensive performance comparison of multiple models, highlighting lightweight models' effectiveness.

Findings

Lightweight models perform comparably to heavyweight models.

Data augmentation improves model performance.

The dataset achieves over 99% accuracy and F1-score.

Abstract

The study involves a comprehensive performance analysis of popular classification and segmentation models, applied over a Bangladeshi pothole dataset, being developed by the authors of this research. This custom dataset of 824 samples, collected from the streets of Dhaka and Bogura performs competitively against the existing industrial and custom datasets utilized in the present literature. The dataset was further augmented four-fold for segmentation and ten-fold for classification evaluation. We tested nine classification models (CCT, CNN, INN, Swin Transformer, ConvMixer, VGG16, ResNet50, DenseNet201, and Xception) and four segmentation models (U-Net, ResU-Net, U-Net++, and Attention-Unet) over both the datasets. Among the classification models, lightweight models namely CCT, CNN, INN, Swin Transformer, and ConvMixer were emphasized due to their low computational requirements and faster prediction times. The lightweight models performed respectfully, oftentimes equating to the performance of heavyweight models. In addition, augmentation was found to enhance the performance of all the tested models. The experimental results exhibit that, our dataset performs on par or outperforms the similar classification models utilized in the existing literature, reaching accuracy and f1-scores over 99%. The dataset also performed on par with the existing datasets for segmentation, achieving model Dice Similarity Coefficient up to 67.54% and IoU scores up to 59.39%.