Edge-Optimized Vision-Language Models for Underground Infrastructure Assessment

TL;DR

This paper introduces an efficient, edge-deployable AI pipeline combining defect segmentation and natural language summarization for underground infrastructure inspection, enabling real-time autonomous assessment on resource-constrained devices.

Contribution

The paper presents a novel two-stage pipeline with a lightweight segmentation model and a fine-tuned vision-language model optimized for edge deployment in underground infrastructure inspection.

Findings

Achieved 0.834 F1-score with 0.64M parameters in defect segmentation.

Enabled real-time summarization with hardware-specific model optimization.

Demonstrated successful deployment on a mobile robotic platform.

Abstract

Autonomous inspection of underground infrastructure, such as sewer and culvert systems, is critical to public safety and urban sustainability. Although robotic platforms equipped with visual sensors can efficiently detect structural deficiencies, the automated generation of human-readable summaries from these detections remains a significant challenge, especially on resource-constrained edge devices. This paper presents a novel two-stage pipeline for end-to-end summarization of underground deficiencies, combining our lightweight RAPID-SCAN segmentation model with a fine-tuned Vision-Language Model (VLM) deployed on an edge computing platform. The first stage employs RAPID-SCAN (Resource-Aware Pipeline Inspection and Defect Segmentation using Compact Adaptive Network), achieving 0.834 F1-score with only 0.64M parameters for efficient defect segmentation. The second stage utilizes a fine-tuned Phi-3.5 VLM that generates concise, domain-specific summaries in natural language from the segmentation outputs. We introduce a curated dataset of inspection images with manually verified descriptions for VLM fine-tuning and evaluation. To enable real-time performance, we employ post-training quantization with hardware-specific optimization, achieving significant reductions in model size and inference latency without compromising summarization quality. We deploy and evaluate our complete pipeline on a mobile robotic platform, demonstrating its effectiveness in real-world inspection scenarios. Our results show the potential of edge-deployable integrated AI systems to bridge the gap between automated defect detection and actionable insights for infrastructure maintenance, paving the way for more scalable and autonomous inspection solutions.