The likely determines the unlikely

TL;DR

This paper introduces a maximum likelihood inference method to determine the frequency distribution of event sizes in complex systems, enabling accurate prediction of rare extreme events from common small events across various domains.

Contribution

The paper presents a novel inference approach that accurately predicts the distribution of rare, large-impact events using minimal observed data, applicable to diverse complex systems.

Findings

Successfully predicts extreme event frequencies in wind speed data.

Accurately estimates the occurrence of rare events like gales and unique character usage.

Demonstrates broad applicability across different complex systems.

Abstract

We point out that the functional form describing the frequency of sizes of events in complex systems (e.g. earthquakes, forest fires, bursts of neuronal activity) can be obtained from maximal likelihood inference, which, remarkably, only involve a few available observed measures such as number of events, total event size and extremes. Most importantly, the method is able to predict with high accuracy the frequency of the rare extreme events. To be able to predict the few, often big impact events, from the frequent small events is of course of great general importance. For a data set of wind speed we are able to predict the frequency of gales with good precision. We analyse several examples ranging from the shortest length of a recruit to the number of Chinese characters which occur only once in a text.