A Multimodal Fusion Framework for Bridge Defect Detection with Cross-Verification

TL;DR

This paper introduces a multimodal fusion framework combining NDE techniques and image processing for improved bridge defect detection, demonstrating promising results in defect localization and validation with an F1 score of 0.83.

Contribution

It presents a novel multimodal fusion framework integrating geospatial analysis and adaptive image processing for bridge defect detection and validation.

Findings

F1 score of 0.83 in defect detection accuracy

Enhanced defect localization and reduced false positives

Effective integration of NDE data with visual analysis

Abstract

This paper presents a pilot study introducing a multimodal fusion framework for the detection and analysis of bridge defects, integrating Non-Destructive Evaluation (NDE) techniques with advanced image processing to enable precise structural assessment. By combining data from Impact Echo (IE) and Ultrasonic Surface Waves (USW) methods, this preliminary investigation focuses on identifying defect-prone regions within concrete structures, emphasizing critical indicators such as delamination and debonding. Using geospatial analysis with alpha shapes, fusion of defect points, and unified lane boundaries, the proposed framework consolidates disparate data sources to enhance defect localization and facilitate the identification of overlapping defect regions. Cross-verification with adaptive image processing further validates detected defects by aligning their coordinates with visual data, utilizing advanced contour-based mapping and bounding box techniques for precise defect identification. The experimental results, with an F1 score of 0.83, demonstrate the potential efficacy of the approach in improving defect localization, reducing false positives, and enhancing detection accuracy, which provides a foundation for future research and larger-scale validation. This preliminary exploration establishes the framework as a promising tool for efficient bridge health assessment, with implications for proactive structural monitoring and maintenance.