A hierarchical semantic segmentation framework for computer vision-based bridge damage detection

TL;DR

This paper presents a hierarchical semantic segmentation framework for bridge damage detection using computer vision, improving detection of small damages and thin objects by leveraging hierarchical relationships, multi-scale augmentation, and importance sampling.

Contribution

The novel framework incorporates hierarchical relationships, multi-scale augmentation, and importance sampling to enhance damage detection accuracy in imbalanced and challenging image datasets.

Findings

Improved detection of small damages like cracks and rebars.

Effective handling of class imbalance through importance sampling.

Enhanced segmentation accuracy with hierarchical context and multi-scale views.

Abstract

Computer vision-based damage detection using remote cameras and unmanned aerial vehicles (UAVs) enables efficient and low-cost bridge health monitoring that reduces labor costs and the needs for sensor installation and maintenance. By leveraging recent semantic image segmentation approaches, we are able to find regions of critical structural components and recognize damage at the pixel level using images as the only input. However, existing methods perform poorly when detecting small damages (e.g., cracks and exposed rebars) and thin objects with limited image samples, especially when the components of interest are highly imbalanced. To this end, this paper introduces a semantic segmentation framework that imposes the hierarchical semantic relationship between component category and damage types. For example, certain concrete cracks only present on bridge columns and therefore the non-column region will be masked out when detecting such damages. In this way, the damage detection model could focus on learning features from possible damaged regions only and avoid the effects of other irrelevant regions. We also utilize multi-scale augmentation that provides views with different scales that preserves contextual information of each image without losing the ability of handling small and thin objects. Furthermore, the proposed framework employs important sampling that repeatedly samples images containing rare components (e.g., railway sleeper and exposed rebars) to provide more data samples, which addresses the imbalanced data challenge.