The 1st Data Science for Pavements Challenge

TL;DR

This paper presents the first Data Science for Pavements Challenge, focusing on developing machine learning algorithms for pavement condition monitoring using benchmarked datasets and innovative data processing techniques.

Contribution

It introduces a data-centric competition framework and reports on top solutions that improved pavement distress detection accuracy through data augmentation and model tuning.

Findings

Top team achieved an F1 score of approximately 0.9

Data cleaning and augmentation significantly improved detection accuracy

Challenges remain in model generalization across diverse conditions

Abstract

The Data Science for Pavement Challenge (DSPC) seeks to accelerate the research and development of automated vision systems for pavement condition monitoring and evaluation by providing a platform with benchmarked datasets and codes for teams to innovate and develop machine learning algorithms that are practice-ready for use by industry. The first edition of the competition attracted 22 teams from 8 countries. Participants were required to automatically detect and classify different types of pavement distresses present in images captured from multiple sources, and under different conditions. The competition was data-centric: teams were tasked to increase the accuracy of a predefined model architecture by utilizing various data modification methods such as cleaning, labeling and augmentation. A real-time, online evaluation system was developed to rank teams based on the F1 score. Leaderboard results showed the promise and challenges of machine for advancing automation in pavement monitoring and evaluation. This paper summarizes the solutions from the top 5 teams. These teams proposed innovations in the areas of data cleaning, annotation, augmentation, and detection parameter tuning. The F1 score for the top-ranked team was approximately 0.9. The paper concludes with a review of different experiments that worked well for the current challenge and those that did not yield any significant improvement in model accuracy.