WhiteTesseract: Reframing the Interpretation of Cultural Heritage through XR and Conversational AI

TL;DR

WhiteTesseract leverages XR and conversational AI to enhance physical cultural heritage exhibitions by enabling personalized, context-aware interpretation that increases visitor engagement and reflection.

Contribution

The paper introduces a novel XR and AI system that improves physical exhibitions by supporting personalized, in-situ interpretation without compromising physical authenticity.

Findings

Viewing duration increased from 35.3 to 98.3 seconds (p < 0.001).

60% of visitor-AI interactions included analytical, emotional, and comparative inquiries.

System effectively supports deeper, personalized engagement in cultural heritage exhibitions.

Abstract

Cultural heritage exhibitions often struggle to sustain attention and support reflective engagement. Physical exhibitions rely on fixed interpretive aids that lack adaptability to individual backgrounds or curiosity, and their effectiveness depends heavily on a visitor's Personal Context, prior knowledge, and cultural literacy. Meanwhile, digital exhibitions prioritize convenience and accessibility but risk weakening the Physical and Social Contexts that define embodied cultural experience. WhiteTesseract addresses this gap by enabling in-situ interpretation through high-resolution XR and conversational AI. The system integrates spatial intelligence via artwork recognition to allow visitors to selectively reduce environmental distractions (via diminished reality) and engage in context-aware dialogue (via large language models). The goal is to preserve the richness of the physical and social environment while providing a flexible space for personal reflection, enhancing Personal Context without compromising physical authenticity. We deployed the system in a Claude Monet exhibition and conducted a controlled user study with 26 participants. Quantitative results showed that WhiteTesseract modulation significantly increased average viewing duration from 35.3 to 98.3 seconds (p < 0.001). Analysis of 529 visitor-AI interactions revealed that 60% extended beyond factual queries to include analytical, emotional, and comparative inquiries. These findings demonstrate how XR and AI can enrich the physical exhibition experience by supporting deeper, more personalized engagement without displacing the embodied value of cultural heritage. We discuss technical and social constraints for real-world deployment and limitations of our controlled setting.