A Method with Feedback for Aggregation of Group Incomplete Pair-Wise Comparisons

TL;DR

This paper introduces a feedback-based method for aggregating incomplete pair-wise comparisons from experts, incorporating hierarchical criteria and expert competence levels to improve decision consistency in strategic planning.

Contribution

It presents a novel combinatorial approach that integrates feedback, hierarchical criteria, and expert competence to enhance aggregation of incomplete pair-wise comparisons.

Findings

Uses double entropy inter-rater index for agreement measurement

Allows experts to choose comparison scales based on their competence

Addresses aggregation in weakly-structured strategic planning domains

Abstract

A method for aggregation of expert estimates in small groups is proposed. The method is based on combinatorial approach to decomposition of pair-wise comparison matrices and to processing of expert data. It also uses the basic principles of Analytic Hierarchy/Network Process approaches, such as building of criteria hierarchy to decompose and describe the problem, and evaluation of objects by means of pair-wise comparisons. It allows to derive priorities based on group incomplete pair-wise comparisons and to organize feedback with experts in order to achieve sufficient agreement of their estimates. Double entropy inter-rater index is suggested for usage as agreement measure. Every expert is given an opportunity to use the scale, in which the degree of detail (number of points/grades) most adequately reflects this expert's competence in the issue under consideration, for every single pair comparison. The method takes all conceptual levels of individual expert competence (subject domain, specific problem, individual pair-wise comparison matrix, separate pair-wise comparison) into consideration. The method is intended to be used in the process of strategic planning in weakly-structured subject domains.