State-of-the-Art Fails in the Art of Damage Detection

TL;DR

This paper highlights the challenges of damage detection in analogue media using machine learning, introduces a comprehensive dataset, and evaluates current models revealing their limitations in generalization.

Contribution

The paper introduces DamBench, a large annotated dataset for damage detection in diverse media, and evaluates various models, exposing their inability to generalize across media types.

Findings

Current models fail to accurately locate damage after supervised training.

Models show limited generalization across different media types.

DamBench provides a valuable resource for future damage detection research.

Abstract

Accurately detecting and classifying damage in analogue media such as paintings, photographs, textiles, mosaics, and frescoes is essential for cultural heritage preservation. While machine learning models excel in correcting global degradation if the damage operator is known a priori, we show that they fail to predict where the damage is even after supervised training; thus, reliable damage detection remains a challenge. We introduce DamBench, a dataset for damage detection in diverse analogue media, with over 11,000 annotations covering 15 damage types across various subjects and media. We evaluate CNN, Transformer, and text-guided diffusion segmentation models, revealing their limitations in generalising across media types.