Designing a Hybrid Neural System to Learn Real-world Crack Segmentation from Fractal-based Simulation

TL;DR

This paper presents a hybrid neural system that uses fractal-based simulation to generate high-quality crack data and learns generalizable features to improve real-world crack segmentation in concrete structures.

Contribution

It introduces a fractal-based crack simulator and a learning framework that bridges simulation and real data for robust crack segmentation.

Findings

Effective handling of real-world crack segmentation

Improved generalization from simulated data

System outperforms existing methods

Abstract

Identification of cracks is essential to assess the structural integrity of concrete infrastructure. However, robust crack segmentation remains a challenging task for computer vision systems due to the diverse appearance of concrete surfaces, variable lighting and weather conditions, and the overlapping of different defects. In particular recent data-driven methods struggle with the limited availability of data, the fine-grained and time-consuming nature of crack annotation, and face subsequent difficulty in generalizing to out-of-distribution samples. In this work, we move past these challenges in a two-fold way. We introduce a high-fidelity crack graphics simulator based on fractals and a corresponding fully-annotated crack dataset. We then complement the latter with a system that learns generalizable representations from simulation, by leveraging both a pointwise mutual information estimate along with adaptive instance normalization as inductive biases. Finally, we empirically highlight how different design choices are symbiotic in bridging the simulation to real gap, and ultimately demonstrate that our introduced system can effectively handle real-world crack segmentation.