Words ranking and Hirsch index for identifying the core of the hapaxes in political texts

TL;DR

This study analyzes rare words in US Presidents' speeches, identifying a core set of hapaxes using rank-size laws and a Hirsch index variant, revealing socio-political insights.

Contribution

It introduces a novel method combining Zipf-Mandelbrot law and a Hirsch index variant to identify the core of hapaxes in political texts.

Findings

The core hapaxes follow a Zipf-Mandelbrot distribution.

Deviations in the distribution reveal socio-political nuances.

The method effectively characterizes rare word usage in speeches.

Abstract

This paper deals with a quantitative analysis of the content of official political speeches. We study a set of about one thousand talks pronounced by the US Presidents, ranging from Washington to Trump. In particular, we search for the relevance of the rare words, i.e. those said only once in each speech -- the so-called hapaxes. We implement a rank-size procedure of Zipf-Mandelbrot type for discussing the hapaxes' frequencies regularity over the overall set of speeches. Starting from the obtained rank-size law, we define and detect the core of the hapaxes set by means of a procedure based on an Hirsch index variant. We discuss the resulting list of words in the light of the overall US Presidents' speeches. We further show that this core of hapaxes itself can be well fitted through a Zipf-Mandelbrot law and that contains elements producing deviations at the low ranks between scatter plots and fitted curve -- the so-called king and vice-roy effect. Some socio-political insights are derived from the obtained findings about the US Presidents messages.