Unveiling Temporal Trends in 19th Century Literature: An Information Retrieval Approach

TL;DR

This study analyzes the evolution of language and themes in 19th century English novels by applying information retrieval techniques to detect temporal shifts in term usage across decades.

Contribution

It introduces a query expansion-based method to quantify linguistic and conceptual changes in literature over time using standard similarity metrics.

Findings

Significant divergence in related terms across decades

Linguistic and thematic shifts are detectable through query expansion

Quantitative measures reveal substantial evolution in 19th century novels

Abstract

In English literature, the 19th century witnessed a significant transition in styles, themes, and genres. Consequently, the novels from this period display remarkable diversity. This paper explores these variations by examining the evolution of term usage in 19th century English novels through the lens of information retrieval. By applying a query expansion-based approach to a decade-segmented collection of fiction from the British Library, we examine how related terms vary over time. Our analysis employs multiple standard metrics including Kendall's tau, Jaccard similarity, and Jensen-Shannon divergence to assess overlaps and shifts in expanded query term sets. Our results indicate a significant degree of divergence in the related terms across decades as selected by the query expansion technique, suggesting substantial linguistic and conceptual changes throughout the 19th century novels.