New results on inconsistency indices and their relationship with the quality of priority vector estimation

TL;DR

This paper investigates the effectiveness of various inconsistency indices in pairwise comparison matrices for estimating priority vectors, clarifying their interpretability and proposing a new inconsistency measure with a statistical acceptance approach.

Contribution

It distinguishes between different types of inconsistency measurement tasks and introduces a new inconsistency index linked to priority vector estimation quality.

Findings

Monte Carlo experiments show the relationship between indices and estimation accuracy

A new inconsistency measure is proposed based on statistical principles

Guidelines for selecting appropriate inconsistency indices in decision making

Abstract

The article is devoted to the problem of inconsistency in the pairwise comparisons based prioritization methodology. The issue of "inconsistency" in this context has gained much attention in recent years. The literature provides us with a number of different "inconsistency" indices suggested for measuring the inconsistency of the pairwise comparison matrix (PCM). The latter is understood as a deviation of the PCM from the "consistent case" - a notion that is formally well-defined in this theory. However the usage of the indices is justified only by some heuristics. It is still unclear what they really "measure". What is even more important and still not known is the relationship between their values and the "consistency" of the decision maker's judgments on one hand, and the prioritization results upon the other. We provide examples showing that it is necessary to distinguish between these three following tasks: the "measuring" of the "PCM inconsistency" and the PCM-based "measuring" of the consistency of decision maker's judgments and, finally, the "measuring" of the usefulness of the PCM as a source of information for estimation of the priority vector (PV). Next we focus on the third task, which seems to be the most important one in Multi-Criteria Decision Making. With the help of Monte Carlo experiments, we study the performance of various inconsistency indices as indicators of the final PV estimation quality. The presented results allow a deeper understanding of the information contained in these indices and help in choosing a proper one in a given situation. They also enable us to develop a new inconsistency characteristic and, based on it, to propose the PCM acceptance approach that is supported by the classical statistical methodology.