Automated Monitoring of Cultural Heritage Artifacts Using Semantic Segmentation

TL;DR

This study compares various U-Net based CNN architectures for pixel-level crack detection in cultural heritage artifacts, demonstrating their effectiveness and generalization in real-world preservation scenarios.

Contribution

It provides a comprehensive evaluation of different CNN encoders for semantic segmentation of cracks on statues and monuments, highlighting their potential for automated heritage preservation.

Findings

Models show promising generalization to unseen artifacts

CNN encoders achieve high segmentation accuracy on test datasets

Out-of-distribution qualitative results indicate robustness

Abstract

This paper addresses the critical need for automated crack detection in the preservation of cultural heritage through semantic segmentation. We present a comparative study of U-Net architectures, using various convolutional neural network (CNN) encoders, for pixel-level crack identification on statues and monuments. A comparative quantitative evaluation is performed on the test set of the OmniCrack30k dataset [1] using popular segmentation metrics including Mean Intersection over Union (mIoU), Dice coefficient, and Jaccard index. This is complemented by an out-of-distribution qualitative evaluation on an unlabeled test set of real-world cracked statues and monuments. Our findings provide valuable insights into the capabilities of different CNN- based encoders for fine-grained crack segmentation. We show that the models exhibit promising generalization capabilities to unseen cultural heritage contexts, despite never having been explicitly trained on images of statues or monuments.