Objective Weights for Scoring: The Automatic Democratic Method

TL;DR

This paper introduces an objective, data-driven method for assigning weights in scoring systems using Data Envelopment Analysis and regression, eliminating personal bias and ensuring fairness.

Contribution

It presents a novel approach combining DEA and regression to determine weights objectively, avoiding subjective judgment in scoring.

Findings

The method accurately recovers known weights in test data.

It produces fair and democratic scoring formulas from data.

The approach is validated on datasets with known true scores and weights.

Abstract

When comparing performance (of products, services, entities, etc.), multiple attributes are involved. This paper deals with a way of weighting these attributes when one is seeking an overall score. It presents an objective approach to generating the weights in a scoring formula which avoids personal judgement. The first step is to find the maximum possible score for each assessed entity. These upper bound scores are found using Data Envelopment Analysis. In the second step the weights in the scoring formula are found by regressing the unique DEA scores on the attribute data. Reasons for using least squares and avoiding other distance measures are given. The method is tested on data where the true scores and weights are known. The method enables the construction of an objective scoring formula which has been generated from the data arising from all assessed entities and is, in that sense, democratic.