Talking to Data: Designing Smart Assistants for Humanities Databases

TL;DR

This paper presents a novel LLM-based smart assistant that enables natural language interaction with humanities databases, improving accessibility and efficiency for researchers and the public.

Contribution

It introduces a chatbot system utilizing RAG and advanced search techniques to facilitate intuitive querying of digital humanities archives.

Findings

The system effectively supports complex database queries via natural language.

Experiments show high response quality across different language models.

The tool enhances accessibility for non-specialist users.

Abstract

Access to humanities research databases is often hindered by the limitations of traditional interaction formats, particularly in the methods of searching and response generation. This study introduces an LLM-based smart assistant designed to facilitate natural language communication with digital humanities data. The assistant, developed in a chatbot format, leverages the RAG approach and integrates state-of-the-art technologies such as hybrid search, automatic query generation, text-to-SQL filtering, semantic database search, and hyperlink insertion. To evaluate the effectiveness of the system, experiments were conducted to assess the response quality of various language models. The testing was based on the Prozhito digital archive, which contains diary entries from predominantly Russian-speaking individuals who lived in the 20th century. The chatbot is tailored to support anthropology and history researchers, as well as non-specialist users with an interest in the field, without requiring prior technical training. By enabling researchers to query complex databases with natural language, this tool aims to enhance accessibility and efficiency in humanities research. The study highlights the potential of Large Language Models to transform the way researchers and the public interact with digital archives, making them more intuitive and inclusive. Additional materials are presented in GitHub repository: https://github.com/alekosus/talking-to-data-intersys2025.