How to Make Museums More Interactive? Case Study of Artistic Chatbot

TL;DR

This paper presents Artistic Chatbot, a voice-based LLM-powered system designed to enhance visitor engagement in museums by providing contextually relevant responses during live exhibitions, demonstrating its potential for increased interactivity.

Contribution

It introduces a novel voice-to-voice RAG-powered chatbot tailored for cultural sites, including system architecture, user interaction design, and deployment insights.

Findings

60% of responses were directly relevant to exhibition content

Chatbot maintained response relevance despite unpredictable queries

Demonstrated potential for increased interactivity in cultural heritage sites

Abstract

Conversational agents powered by Large Language Models (LLMs) are increasingly utilized in educational settings, in particular in individual closed digital environments, yet their potential adoption in the physical learning environments like cultural heritage sites, museums, and art galleries remains relatively unexplored. In this study, we present Artistic Chatbot, a voice-to-voice RAG-powered chat system to support informal learning and enhance visitor engagement during a live art exhibition celebrating the 15th anniversary of the Faculty of Media Art at the Warsaw Academy of Fine Arts, Poland. The question answering (QA) chatbot responded to free-form spoken questions in Polish using the context retrieved from a curated, domain-specific knowledge base consisting of 226 documents provided by the organizers, including faculty information, art magazines, books, and journals. We describe the key aspects of the system architecture and user interaction design, as well as discuss the practical challenges associated with deploying chatbots at public cultural sites. Our findings, based on interaction analysis, demonstrate that chatbots such as Artistic Chatbot effectively maintain responses grounded in exhibition content (60\% of responses directly relevant), even when faced with unpredictable queries outside the target domain, showing their potential for increasing interactivity in public cultural sites. GitHub project page: https://github.com/cinekucia/artistic-chatbot-cikm2025

Tables

Table 1. Table 1: Interaction Statistics: Question (Q.) completeness, Question relevance and Response (R.) relevance to the exhibition domain.

Metric	Yes (#)	No (#)	Yes (%)
Q. completeness	$497$	$230$	68.37
Q. relevance	$142$	$585$	$19.53$
R. relevance	$440$	$287$	$60.52$

Table 2. Table 2: Question-response relevance score distribution (Mean: 2.66 2.66 ). Note the scale: 1 (not relevant) to 5 (very relevant)

Score	$1$	$2$	$3$	$4$	$5$
Count	$286$	$37$	$169$	$110$	$125$
Percentage (%)	$39.4$	$5.1$	$23.2$	$15.1$	$17.2$

Full text

During the demo presentation, the audience will be invited to query our Artistic Chatbot, which adopts the persona of an artificial art curator, a role that involves responding to questions while simultaneously assessing their relevance to the exhibition. The link for the demo video is available here.

K****eywords Natural Language Processing  $\cdot$ Voice interface  $\cdot$ Human-Computer Interaction  $\cdot$ Large Language Model

1 Introduction

The rapid advancements in Artificial Intelligence (AI), and specifically Natural Language Processing (NLP), led to the widespread adoption of Large Language Models (LLMs) [1]. While LLMs can be used in a variety of settings, they are particularly well-suited for the development of conversational agents, also known as chatbots and voice assistants [2, 3]. Such systems, powered by state-of-the-art models like GPT, have proven highly effective in education [4], where they are fundamental in the development of intelligent tutoring systems and personalized learning platforms. By leveraging their ability to generate meaningful human-like responses, these assistants greatly enhance user interactions and offer much more tailored and context-grounded conversations [5].

Most educational applications of LLM-powered chatbots focus on individualized learning scenarios [6, 7], that is, systems used in isolated, digital environments, and interaction via text-based interfaces. However, relatively little research explores the potential of conversational agents in more social, shared, and interactive settings such as classrooms, or cultural and public education venues, including exhibition sites at museums and galleries [8, 9].

Several previous works highlight the effectiveness of using chatbots in cultural heritage and museum sites for enhancing visitor engagement and easier access to information [10, 11]. A chatbot system deployed in archaeological sites in Pompeii (Italy) used semantic analysis to extract topics from written user queries, contextualize them with information from third-party web services (e.g., Wikipedia, TripAdvisor), and a static knowledge base to formulate answers to visitors’ questions [12, 13]. Another study approached contextual visual question answering with the use of Multi-modal Large Language Models (MLLMs) [14]. The system combined visual inputs with the relevant artwork descriptions from a database of Wikipedia content, ensuring scientifically accurate information is provided.

In this work, we demonstrate Artistic Chatbot, a voice-to-voice Question-Answering (QA) chatbot developed for and deployed during the exhibition commemorating the 15th anniversary of the Faculty of Media Art at the Warsaw Academy of Fine Arts, Poland. The system allows visitors to ask free-form spoken questions about the faculty, artists, exhibitions, and pieces of art, and receive human-like spoken answers, which are grounded in the domain-specific corpus provided beforehand by the organizers.

2 System Design and Implementation

The development of the chatbot involved two stages: a data preprocessing phase to construct a specialized knowledge base, and an inference pipeline designed to handle user interactions.

2.1 Data Preprocessing

The initial stage focused on preparing the knowledge base from a corpus of $226$ ($159$ raw PDF documents and $67$ artists’ and academy employees’ biographies) provided by the Academy of Fine Arts in Warsaw (see Figure 1). These source documents presented considerable challenges, primarily due to complex layouts, the presence of multiple languages — such as Polish, Mandarin Chinese, English, German, and French — and inherent inconsistencies common in unstructured data. While about 92% of the data was in Polish, many documents included fragments in the other languages, often within the same page, which added to the overall complexity of interpretation and processing.

To refine this raw data into a high-quality, consistent resource, the extracted text files were cleaned and translated into Polish using a multilingual Large Language Model – GPT-4o [15].

The dataset documents were segmented into $11,596$ smaller, overlapping chunks to facilitate effective Retrieval-Augmented Generation (RAG) [16]. The chunks were capped at $5,000$ characters, with an overlap of $200$ characters to ensure contextual continuity between segments. On average, each document was split into approximately $51$ chunks (median: $19.5$), though the number varied depending on length and structure (ranging from $1$ to $650$ chunks per document). Each generated text chunk was then transformed into a dense vector representation (embedding) using the sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 model. Subsequently, these embeddings were indexed into a FAISS (Facebook AI Similarity Search) [17] vector store for fast lookup of the most relevant chunks.

2.2 Inference Pipeline

The second stage, the inference pipeline, was key to the user interaction flow (see Figure 2). User engagement starts with voice input. The system actively listened for predefined trigger phrases (see Section 2.3) before capturing the user’s query. This allowed us to avoid undesired triggers and clearly indicated the start of the query. Upon successful query transcription into text, the RAG mechanism was invoked. This involved a two-step retrieval process. First, the user’s query was embedded. The FAISS vector store then performed an initial similarity search, retrieving the top $20$ potentially relevant chunks from the indexed knowledge base. This initial retrieval prioritized speed over fine-grained relevance; thus, we needed a subsequent re-ranking step. It employed a pre-trained CrossEncoder model cross-encoder/ms-marco-MiniLM-L12-v2, which evaluated the relevance of each of the $20$ retrieved passages against the original query more precisely. Only the top $3$ highest-scoring chunks, as determined by the re-ranker, were selected to form the contextual basis for the final response generation. The selected context, along with the user’s query, was then used to construct a prompt for the LLM. The response generation was handled by the GPT-4o-mini model. The prompt included a dynamically selected system message, with a response style chosen randomly at the start of the session, including ’normal’, ’academic’, or ’laid-back’ templates. This system prompt defined the chatbot’s persona to differentiate the system’s style of answers. The prompt also provided crucial contextual information about the exhibition, such as its location, exhibition period, and outlined specific conversational guidelines, including responding only in Polish and avoiding direct reference to the retrieved context. Finally, the text response was synthesized back into audible speech using the ElevenLabs Text-to-Speech API. Each complete interaction, including the timestamp, user input query, the style of the system prompt used, and the final generated response, was systematically logged to enable continuous monitoring and subsequent analysis of the chatbot’s performance.

2.3 User Interaction

The exhibition physical setup included a ceiling-mounted microphone, located in the center of the room, and four speakers installed in the corners. The peripherals were connected to a nearby PC station running the chatbot and logging user responses in real-time (see Figure 3).

To initiate interaction with the Artistic Chatbot, visitors were required to walk up to the suspended microphone and articulate one of the four designated trigger expressions (“Hello,” “Welcome,” “Question,” or “I have a question”222Translated from Polish: “Cześć”, “Witaj”, “Pytanie”, “Mam pytanie”). Upon recognizing a trigger, the system would greet the visitor, thereby signaling its readiness to receive a question. This mechanism ensured that the input was intentional, distinguishing user interaction from background noise.

The primary aim was to encourage visitors to ask questions related to the exhibition, the participating artists, or the history of the Faculty of Media Art. Once greeted, users could pose virtually any question within that thematic scope. The chatbot was designed for single-turn interactions, meaning it responded to one question at a time without retaining contextual information from previous exchanges. Before accepting a new question, the chatbot had to finish articulating the response, which was generated by the LLM and served in audio form. Visitors were expected to wait until the spoken response had been completely delivered before initiating another query. This constraint ensured clarity, prevented interruptions, and maintained the coherence of the interaction.

After each question-and-answer cycle, the system automatically returned to its initial state, listening for trigger phrases to reactivate the chatbot. This interaction model enabled a direct, accessible way for users to engage with the content and concept of the exhibition by facilitating simulated conversations with an artificial art curator through voice input.

3 Interaction Analysis

A total of $727$ questions were asked over the course of the month in which the exhibition was open to the public. We evaluated the questions and the LLM’s responses utilizing LLM-as-a-judge technique (following [18]). We prompted an LLM to check the completeness of the questions, expand them, rate the relevance of the responses, and categorize the questions. One of the main challenges was question completeness (see Table 1). Nearly one-third of all captured user queries were incomplete or prematurely cut off. This issue largely stemmed from the system’s reliance on silence-based end-of-utterance detection. However, the analysis showed that correcting the questions usually required only minor edits – on average about 3.4 characters (within the range 1 – 6, measured by Levenshtein distance).

As in other exhibition settings, e.g., in  [19], the majority of questions – about $600$ – were categorized as simple factual questions, followed by about $150$ casual and confirmation questions, and $24$ hypothetical questions (e.g., “what would happen if X was true”). This behavioral pattern heavily reduced the number of meaningful interactions with the system and was identified as a key area for improvement. Our visitors frequently strayed from the intended scope of questions, which is a common challenge in public, voice-based installations. In fact, only one-fifth of the questions were fully related to the target domain (see Table 1). However, as the system consistently supplied the LLM with exhibition-related context retrieved from the knowledge base, many answers remained focused on the exhibition, even when the user’s questions were unrelated (see Table 2).

The results point to the system’s strength in grounding the responses in the target context, which was of particular value for the exhibition. Nevertheless, we identified a need for further development and measurements in input handling and the relevance of responses, especially in open public settings where high variability of user utterances is expected.

4 Limitations & Ethical Considerations

The Artistic Chatbot served as a live prototype, and naturally involved some practical shortcuts, resulting in several limitations.

Firstly, the dataset utilized for RAG was relatively small. Expanding the dataset is one solution, yet it might affect retrieval latency and thus the overall system efficiency. These trade-offs, however, are considered inevitable and typical for RAG model inference [20]. That is why designing scalable chatbots in similar settings should focus on effective context utilization through optimal chunking [21] and proper choice of RAG system parameters, for instance, the number of retrieved chunks [22]. Additionally, the translation of the source materials into Polish could potentially introduce inaccuracies, mainly due to domain terminology and cultural context shifts. As art collections typically span numerous languages, alternative approaches may be used instead of fully translation-based RAG pipelines, for example, Cross-lingual RAG [23], where retrieval is performed in the original language and only the most relevant chunks are translated prior to response generation.

Secondly, a few User Experience (UX) issues were observed. The system relies on detecting pauses to determine the end of a user’s query, which may lead to premature termination of user input retrieval, particularly in noisy environments or when the user hesitates while speaking. Changing the interaction from voice-command activation to physical button activation to make the system listen to the user is a simple solution to this problem. An alternative approach, without altering the physical interaction interface, could involve enhancing the existing silence detection with a more sophisticated end-of-utterance (EoU) [24] validation mechanism.

Deployment of LLM-powered chat systems in public places, and especially in museums and cultural heritage sites, requires addressing potential ethical issues. One of the primary concerns is the truthfulness of LLMs and RAG systems. LLMs can produce hallucinations, often giving plausible yet false or misleading information. While domain RAG reduces the probability of hallucinations, false information may still occur due to inaccuracies in source documents or LLM’s misinterpretation of context [25]. To prevent hallucination and increase the chatbot’s reliability, one may consider fine-tuning the model to acknowledge uncertainty, that is, responding with "I don’t know" when needed [26]. Another ethical aspect to consider is the safety of chatbot responses. Unless content filtering and moderation techniques are employed, there exists an increased risk of generating harmful or inappropriate content when the system is misused or abused by users, for example, via prompt engineering.

5 Conclusions

The deployment of Artistic Chatbot at a public art exhibition demonstrated the potential of LLM-powered, voice-based agents to enhance interactivity and visitor engagement in cultural heritage settings. The chatbot was successfully run for a month as part of the exhibition. Over $727$ queries were recorded during the month-long event, with more than $60\%$ of answers judged as related to the exhibition, despite only around $20\%$ of user questions being strictly on-topic. This shows the system’s ability to keep responses grounded in exhibition content, even when faced with unpredictable input that is out of the target domain. However, retrieval-based responses also introduced trade-offs, such as limited flexibility and reduced answer diversity when handling vague or ambiguous queries. Challenges like human-chatbot interaction design, simplistic end-of-utterance detection, and moderate overall response relevance (mean score: $2.66$) are identified as key areas for future improvement.

Acknowledgments

We would like to thank our partners at ElevenLabs for providing Text-to-Speech service and the employees of the Academy of Fine Arts for their collaboration.