Augmented Reality in Cultural Heritage: A Dual-Model Pipeline for 3D Artwork Reconstruction

TL;DR

This paper introduces a novel augmented reality pipeline for museum artworks that combines two pre-trained depth estimation models to produce accurate 3D reconstructions from single images, enhancing visitor engagement.

Contribution

It presents a dual-model approach integrating global and local depth estimation models for improved 3D artwork reconstruction in AR applications.

Findings

Significant improvements in reconstruction accuracy.

Enhanced visual realism of 3D models.

Robustness to irregular contours and textures.

Abstract

This paper presents an innovative augmented reality pipeline tailored for museum environments, aimed at recognizing artworks and generating accurate 3D models from single images. By integrating two complementary pre-trained depth estimation models, i.e., GLPN for capturing global scene structure and Depth-Anything for detailed local reconstruction, the proposed approach produces optimized depth maps that effectively represent complex artistic features. These maps are then converted into high-quality point clouds and meshes, enabling the creation of immersive AR experiences. The methodology leverages state-of-the-art neural network architectures and advanced computer vision techniques to overcome challenges posed by irregular contours and variable textures in artworks. Experimental results demonstrate significant improvements in reconstruction accuracy and visual realism, making the system a highly robust tool for museums seeking to enhance visitor engagement through interactive digital content.