Weakly Supervised SVM-Enhanced SAM Pipeline for Stone-by-Stone Segmentation of the Masonry of the Loire Valley Castles

TL;DR

This paper introduces a new computer vision method to automatically map stones in historic castles, improving restoration efforts.

Contribution

A novel weakly supervised pipeline combining SAM and SVM for stone segmentation in masonry is proposed.

Findings

The proposed SAM-SVM architecture achieves an 85% Dice coefficient for stone segmentation.

The method is scalable and efficient for cultural heritage conservation tasks.

Results are validated through extensive experimentation and computer vision evaluation.

Abstract

The preservation of historical monuments presents a formidable challenge, particularly in monitoring the deterioration of building materials over time. Chateau de Chambord’s facade suffers from common issues such as flaking and spalling, which require meticulous stone and joint mapping from experts manually for restoration efforts. Advancements in computer vision have allowed machine-learning models to help in the automatic segmentation process. In this research, a custom architecture defined as SAM-SVM is proposed, to perform stone segmentation, based on the Segment Anything Model (SAM) and Support Vector Machines (SVM). By exploiting the zero-shot learning capabilities of SAM and its customizable input parameters, we obtain segmentation mask for stones and joints, which are then classified using SVM. Two more SAMs (three in total) are used, depending on how many stones are left to segment. Through extensive experimentation and evaluation, supported by computer vision methods, the proposed architecture achieves a Dice coefficient of 85%. Our results highlight the potential of SAM in cultural heritage conservation, providing a scalable and efficient solution for stone segmentation in historic monuments. This research contributes valuable insights and methodologies to the ongoing conservation efforts of Château de Chambord and could be extrapolated to other monuments.

Figures