Historia Magistra Vitae: Dynamic Topic Modeling of Roman Literature using Neural Embeddings

TL;DR

This paper explores the use of BERT-based neural embeddings for dynamic topic modeling of Roman literature, showing it provides better qualitative insights and is more user-friendly than traditional statistical models despite lower quantitative scores.

Contribution

It introduces a BERT-based neural dynamic topic model for historical texts and compares its effectiveness and usability against traditional models like LDA and NMF.

Findings

Neural model yields better qualitative insights.

Neural model is less sensitive to hyperparameters.

Traditional models have higher quantitative metrics.

Abstract

Dynamic topic models have been proposed as a tool for historical analysis, but traditional approaches have had limited usefulness, being difficult to configure, interpret, and evaluate. In this work, we experiment with a recent approach for dynamic topic modeling using BERT embeddings. We compare topic models built using traditional statistical models (LDA and NMF) and the BERT-based model, modeling topics over the entire surviving corpus of Roman literature. We find that while quantitative metrics prefer statistical models, qualitative evaluation finds better insights from the neural model. Furthermore, the neural topic model is less sensitive to hyperparameter configuration and thus may make dynamic topic modeling more viable for historical researchers.