Robotic Defect Inspection with Visual and Tactile Perception for Large-scale Components

TL;DR

This paper introduces a two-stage multi-modal inspection system combining visual and tactile sensing to detect surface defects on large, curved industrial components, achieving high defect detection rates in real-world aerospace manufacturing settings.

Contribution

The paper presents a novel multi-modal inspection pipeline and a new real-world dataset for defect detection on large-scale components, improving detection accuracy in complex industrial environments.

Findings

Achieved 85% defect detection after Stage I

Achieved 100% defect detection after Stage II

Proposed a publicly available dataset for benchmarking

Abstract

In manufacturing processes, surface inspection is a key requirement for quality assessment and damage localization. Due to this, automated surface anomaly detection has become a promising area of research in various industrial inspection systems. A particular challenge in industries with large-scale components, like aircraft and heavy machinery, is inspecting large parts with very small defect dimensions. Moreover, these parts can be of curved shapes. To address this challenge, we present a 2-stage multi-modal inspection pipeline with visual and tactile sensing. Our approach combines the best of both visual and tactile sensing by identifying and localizing defects using a global view (vision) and using the localized area for tactile scanning for identifying remaining defects. To benchmark our approach, we propose a novel real-world dataset with multiple metallic defect types per image, collected in the production environments on real aerospace manufacturing parts, as well as online robot experiments in two environments. Our approach is able to identify 85% defects using Stage I and identify 100% defects after Stage II. The dataset is publicly available at https://zenodo.org/record/8327713