A Hybrid YOLO and Segment Anything Model Pipeline for Multi-Damage Segmentation in UAV Inspection Imagery
Rafael Cabral, Ricardo Santos, José A. F. O. Correia, Diogo Ribeiro
TL;DR
This paper proposes a hybrid model combining YOLO and SAM to improve multi-damage segmentation in UAV images for infrastructure inspection.
Contribution
A class-specific hybrid pipeline that leverages YOLO and SAM for accurate multi-damage segmentation in UAV imagery.
Findings
The hybrid pipeline achieved a mean Average Precision (mAP50) of 0.593.
Class-specific Intersection over Union (IoU) scores were 0.495 for cracks, 0.331 for efflorescence, and 0.205 for exposed rebar.
The results highlight the effectiveness of combining specialized detectors with foundation models for infrastructure inspection.
Abstract
The automated inspection of civil infrastructure with Unmanned Aerial Vehicles (UAVs) is hampered by the challenge of accurately segmenting multi-damage in high-resolution imagery. While foundational models like the Segment Anything Model (SAM) offer data-efficient segmentation, their effectiveness is constrained by prompting strategies, especially for geometrically complex defects. This paper presents a comprehensive comparative analysis of deep learning strategies to identify an optimal deep learning pipeline for segmenting cracks, efflorescences, and exposed rebars. It systematically evaluates three distinct end-to-end segmentation frameworks: the native output of a YOLO11 model; the Segment Anything Model (SAM), prompted by bounding boxes; and SAM, guided by a point-prompting mechanism derived from the detector’s probability map. Based on these findings, a final, optimized hybrid…
Genes, proteins, chemicals, diseases, species, mutations and cell lines named across the full text — each resolved to its canonical identifier and authoritative record.
Click any figure to enlarge with its caption.
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13
Figure 14
Figure 15Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsInfrastructure Maintenance and Monitoring · Advanced Neural Network Applications · Concrete Corrosion and Durability