Integrating Visual and X-Ray Machine Learning Features in the Study of Paintings by Goya

TL;DR

This paper presents a multimodal machine learning framework that combines visual and X-ray features with identical extraction techniques to improve the authentication of Goya's paintings, achieving high accuracy and practical efficacy.

Contribution

The study introduces a novel unified feature extraction pipeline applied to both visual and X-ray images for art authentication, demonstrating improved performance over single-modal methods.

Findings

Achieved 97.8% classification accuracy on Goya paintings dataset.

Demonstrated 92.3% authentication confidence in a case study.

Substantial performance gains over single-modal approaches.

Abstract

Art authentication of Francisco Goya's works presents complex computational challenges due to his heterogeneous stylistic evolution and extensive historical patterns of forgery. We introduce a novel multimodal machine learning framework that applies identical feature extraction techniques to both visual and X-ray radiographic images of Goya paintings. The unified feature extraction pipeline incorporates Grey-Level Co-occurrence Matrix descriptors, Local Binary Patterns, entropy measures, energy calculations, and colour distribution analysis applied consistently across both imaging modalities. The extracted features from both visual and X-ray images are processed through an optimised One-Class Support Vector Machine with hyperparameter tuning. Using a dataset of 24 authenticated Goya paintings with corresponding X-ray images, split into an 80/20 train-test configuration with 10-fold cross-validation, the framework achieves 97.8% classification accuracy with a 0.022 false positive rate. Case study analysis of ``Un Gigante'' demonstrates the practical efficacy of our pipeline, achieving 92.3% authentication confidence through unified multimodal feature analysis. Our results indicate substantial performance improvement over single-modal approaches, establishing the effectiveness of applying identical computational methods to both visual and radiographic imagery in art authentication applications.