CNN Model & Tuning for Global Road Damage Detection

TL;DR

This paper evaluates and benchmarks different CNN architectures and tuning strategies for the Global Road Damage Detection Challenge, demonstrating the effectiveness of a generalizable Resnet-50 based Faster R-CNN model across diverse datasets.

Contribution

The study compares single and multi-stage object detection architectures, providing a benchmark and insights into model generalizability for road damage detection across multiple countries.

Findings

Resnet-50 Faster R-CNN achieved a mean F1 score of 0.542.

Resnet-50 showed better generalizability than more complex models.

Benchmark results using Detectron2 and Yolov5 frameworks.

Abstract

This paper provides a report on our solution including model selection, tuning strategy and results obtained for Global Road Damage Detection Challenge. This Big Data Cup Challenge was held as a part of IEEE International Conference on Big Data 2020. We assess single and multi-stage network architectures for object detection and provide a benchmark using popular state-of-the-art open-source PyTorch frameworks like Detectron2 and Yolov5. Data preparation for provided Road Damage training dataset, captured using smartphone camera from Czech, India and Japan is discussed. We studied the effect of training on a per country basis with respect to a single generalizable model. We briefly describe the tuning strategy for the experiments conducted on two-stage Faster R-CNN with Deep Residual Network (Resnet) and Feature Pyramid Network (FPN) backbone. Additionally, we compare this to a one-stage Yolov5 model with Cross Stage Partial Network (CSPNet) backbone. We show a mean F1 score of 0.542 on Test2 and 0.536 on Test1 datasets using a multi-stage Faster R-CNN model, with Resnet-50 and Resnet-101 backbones respectively. This shows the generalizability of the Resnet-50 model when compared to its more complex counterparts. Experiments were conducted using Google Colab having K80 and a Linux PC with 1080Ti, NVIDIA consumer grade GPU. A PyTorch based Detectron2 code to preprocess, train, test and submit the Avg F1 score to is made available at https://github.com/vishwakarmarhl/rdd2020