Learning to Automatically Catch Potholes in Worldwide Road Scene Images

TL;DR

This paper presents a deep learning-based system for automatic pothole detection in diverse real-world road scene images, leveraging a large annotated dataset and deploying the model on embedded vehicle hardware for real-time hazard notification.

Contribution

It introduces a novel application of advanced AI object detection models trained on a diverse global dataset for pothole identification in real-time vehicle systems.

Findings

High average precision in pothole detection

Successful deployment on Nvidia DrivePX2 platform

Real-time notification to IoT platform

Abstract

Among several road hazards that are present in any paved way in the world, potholes are one of the most annoying and also involving higher maintenance costs. There exists an increasing interest on the automated detection of these hazards enabled by technological and research progress. Our research work tackled the challenge of pothole detection from images of real world road scenes. The main novelty resides on the application of the latest progress in AI to learn the visual appearance of potholes. We built a large dataset of images with pothole annotations. They contained road scenes from different cities in the world, taken with different cameras, vehicles and viewpoints under varied environmental conditions. Then, we fine-tuned four different object detection models based on Faster R-CNN and SSD deep neural networks. We achieved high average precision and the pothole detector was tested on the Nvidia DrivePX2 platform with GPGPU capability, which can be embedded on vehicles. Moreover, it was deployed on a real vehicle to notify the detected potholes to a given IoT platform as part of AUTOPILOT H2020 project.