Transmission Line Defect Detection Based on UAV Patrol Images and Vision-language Pretraining

TL;DR

This paper introduces a novel vision-language pretraining approach combined with a progressive transfer strategy to enhance defect detection accuracy in UAV patrol images of transmission lines, addressing issues caused by limited visual information.

Contribution

It proposes a specialized vision-language pretraining method and a progressive transfer strategy tailored for transmission line defect detection in UAV images, improving detection performance.

Findings

Significant improvement in defect detection accuracy.

Effective use of multimodal information from visual and linguistic data.

Overcoming visual information limitations in UAV images.

Abstract

Unmanned aerial vehicle (UAV) patrol inspection has emerged as a predominant approach in transmission line monitoring owing to its cost-effectiveness. Detecting defects in transmission lines is a critical task during UAV patrol inspection. However, due to imaging distance and shooting angles, UAV patrol images often suffer from insufficient defect-related visual information, which has an adverse effect on detection accuracy. In this article, we propose a novel method for detecting defects in UAV patrol images, which is based on vision-language pretraining for transmission line (VLP-TL) and a progressive transfer strategy (PTS). Specifically, VLP-TL contains two novel pretraining tasks tailored for the transmission line scenario, aimimg at pretraining an image encoder with abundant knowledge acquired from both visual and linguistic information. Transferring the pretrained image encoder to the defect detector as its backbone can effectively alleviate the insufficient visual information problem. In addition, the PTS further improves transfer performance by progressively bridging the gap between pretraining and downstream defection detection. Experimental results demonstrate that the proposed method significantly improves defect detection accuracy by jointly utilizing multimodal information, overcoming the limitations of insufficient defect-related visual information provided by UAV patrol images.