Approaches to Analysing Historical Newspapers Using LLMs

TL;DR

This paper combines topic modeling, LLM-based sentiment analysis, and discourse analysis to study Slovene historical newspapers, revealing ideological differences and identity portrayals at the turn of the twentieth century.

Contribution

It introduces a mixed methods approach integrating computational techniques and critical discourse analysis for analyzing noisy historical newspaper data.

Findings

Identified thematic patterns reflecting ideological orientations.

Selected GaMS3-12B-Instruct as the best LLM for sentiment analysis in degraded OCR data.

Revealed variation in portrayal of identities through entity-graph analysis.

Abstract

This study presents a computational analysis of the Slovene historical newspapers \textit{Slovenec} and \textit{Slovenski narod} from the sPeriodika corpus, combining topic modelling, large language model (LLM)-based aspect-level sentiment analysis, entity-graph visualisation, and qualitative discourse analysis to examine how collective identities, political orientations, and national belonging were represented in public discourse at the turn of the twentieth century. Using BERTopic, we identify major thematic patterns and show both shared concerns and clear ideological differences between the two newspapers, reflecting their conservative-Catholic and liberal-progressive orientations. We further evaluate four instruction-following LLMs for targeted sentiment classification in OCR-degraded historical Slovene and select the Slovene-adapted GaMS3-12B-Instruct model as the most suitable for large-scale application, while also documenting important limitations, particularly its stronger performance on neutral sentiment than on positive or negative sentiment. Applied at dataset scale, the model reveals meaningful variation in the portrayal of collective identities, with some groups appearing predominantly in neutral descriptive contexts and others more often in evaluative or conflict-related discourse. We then create NER graphs to explore the relationships between collective identities and places. We apply a mixed methods approach to analyse the named entity graphs, combining quantitative network analysis with critical discourse analysis. The investigation focuses on the emergence and development of intertwined historical political and socionomic identities. Overall, the study demonstrates the value of combining scalable computational methods with critical interpretation to support digital humanities research on noisy historical newspaper data.