A simple tool for weighted averaging of inconsistent data sets

TL;DR

This paper introduces a Bayesian-based weighted averaging method for inconsistent data sets, offering a robust, general approach with a Python implementation, applicable to scientific data with outliers and uncertainties.

Contribution

It presents a simple, general Bayesian method for weighted averaging of inconsistent data, with a Python library for easy application in scientific research.

Findings

Effective handling of outliers and scattered data

Application to various critical data sets including fundamental constants

Robustness demonstrated across multiple scientific examples

Abstract

The weighted average of inconsistent data is a common and tedious problem that many scientists have encountered. The standard weighted average is not recommended for these cases, and various alternative methods have been proposed. These approaches vary in suitability depending on the nature of the data, which can make selecting the appropriate method difficult without expertise in metrology or statistics. For the analysis of simple data sets presenting inconsistencies, we discuss the method proposed by Sivia in 1996 based on Bayesian statistics. This choice has the intention of maintaining generality while minimising the number of assumptions. In this approach, the uncertainty associated with each input value is considered to be just a lower bound of the true unknown uncertainty. The resulting likelihood function is no longer Gaussian but has smoothly decreasing wings, which allows for a better treatment of scattered data and outliers. To demonstrate the robustness and the generality of the method, we apply it to a series of critical data sets: simulations, CODATA recommended values of the Newtonian gravitational constant, and some particle properties from the Particle Data Group, including the proton charge radius. A freely available Python library is also provided for a simple implementation of the proposed averaging method.