Comparative Analysis of Custom CNN Architectures versus Pre-trained Models and Transfer Learning: A Study on Five Bangladesh Datasets

TL;DR

This paper compares custom CNNs and pre-trained models with transfer learning on five Bangladesh datasets, showing transfer learning generally yields higher accuracy, especially on complex tasks, with ResNet-18 achieving perfect accuracy on one dataset.

Contribution

It provides a comprehensive comparison of custom CNNs versus pre-trained models with transfer learning across diverse datasets, highlighting the advantages of transfer learning for complex tasks.

Findings

Transfer learning with fine-tuning outperforms custom CNNs in accuracy.

ResNet-18 achieved 100% accuracy on Road Damage BD dataset.

Custom CNNs are more efficient in size and training on simpler tasks.

Abstract

This study presents a comprehensive comparative analysis of custom-built Convolutional Neural Networks (CNNs) against popular pre-trained architectures (ResNet-18 and VGG-16) using both feature extraction and transfer learning approaches. We evaluated these models across five diverse image classification datasets from Bangladesh: Footpath Vision, Auto Rickshaw Detection, Mango Image Classification, Paddy Variety Recognition, and Road Damage Detection. Our experimental results demonstrate that transfer learning with fine-tuning consistently outperforms both custom CNNs built from scratch and feature extraction methods, achieving accuracy improvements ranging from 3% to 76% across different datasets. Notably, ResNet-18 with fine-tuning achieved perfect 100% accuracy on the Road Damage BD dataset. While custom CNNs offer advantages in model size (3.4M parameters vs. 11-134M for pre-trained models) and training efficiency on simpler tasks, pre-trained models with transfer learning provide superior performance, particularly on complex classification tasks with limited training data. This research provides practical insights for practitioners in selecting appropriate deep learning approaches based on dataset characteristics, computational resources, and performance requirements.