Vision-Guided Active Tactile Perception for Crack Detection and Reconstruction

TL;DR

This paper introduces a novel vision-guided active tactile perception method for detecting and reconstructing cracks in concrete structures, enhancing robustness over vision-only approaches and aiding autonomous infrastructure inspection.

Contribution

It presents a new integrated approach combining vision and tactile sensing for more accurate and robust crack detection and reconstruction in challenging environments.

Findings

Improved crack detection accuracy over vision-only methods

Enhanced robustness against lighting and shadow variances

Potential for autonomous infrastructure inspection and repair

Abstract

Crack detection is of great significance for monitoring the integrity and well-being of the infrastructure such as bridges and underground pipelines, which are harsh environments for people to access. In recent years, computer vision techniques have been applied in detecting cracks in concrete structures. However, they suffer from variances in light conditions and shadows, lacking robustness and resulting in many false positives. To address the uncertainty in vision, human inspectors actively touch the surface of the structures, guided by vision, which has not been explored in autonomous crack detection. In this paper, we propose a novel approach to detect and reconstruct cracks in concrete structures using vision-guided active tactile perception. Given an RGB-D image of a structure, the rough profile of the crack in the structure surface will first be segmented with a fine-tuned Deep Convolutional Neural Networks, and a set of contact points are generated to guide the collection of tactile images by a camera-based optical tactile sensor. When contacts are made, a pixel-wise mask of the crack can be obtained from the tactile images and therefore the profile of the crack can be refined by aligning the RGB-D image and the tactile images. Extensive experiment results have shown that the proposed method improves the effectiveness and robustness of crack detection and reconstruction significantly, compared to crack detection with vision only, and has the potential to enable robots to help humans with the inspection and repair of the concrete infrastructure.