Classifying Inconsistency in AHP Pairwise Comparison Matrices Using Machine Learning

TL;DR

This paper introduces a machine learning-based method using triadic preference reversals to assess consistency in AHP pairwise comparison matrices, achieving higher accuracy than traditional methods.

Contribution

It proposes a novel triadic preference reversal approach combined with clustering and classification for more reliable AHP consistency assessment.

Findings

PR method achieves 97% accuracy in detecting inconsistency

Outperforms the traditional CR method with 50% accuracy

Implemented in an accessible R package on CRAN

Abstract

Assessing consistency in Pairwise Comparison Matrices (PCMs) within the Analytical Hierarchy Process (AHP) poses significant challenges when using the traditional Consistency Ratio (CR) method. This study introduces a novel alternative that leverages triadic preference reversals (PR) to provide a more robust and interpretable assessment of consistency. Triadic preference reversals capture inconsistencies between a pair of elements by comparing the direction of preference derived from the global eigenvector with that from a 3x3 submatrix (triad) containing the same pair, highlighting local-global preference conflicts. This method detects a reversal when one eigen ratio exceeds one while another falls below one, signaling inconsistency. We identify two key features: the proportion of preference reversals and the maximum reversal, which mediate the impact of a PCM's order on its consistency. Using these features simulated PCMs are clustered into consistent and inconsistent classes through k-means clustering, followed by training a logistic classifier for consistency evaluation. The PR method achieves 97\% accuracy, significantly surpassing the Consistency Ratio (CR) method's 50%, with a false negative rate of only 2.6\% compared to 5.5\%. These findings demonstrate the PR method's superior accuracy in assessing AHP consistency, thereby enabling more reliable decision-making. The proposed triadic preference reversal (PR) approach is implemented in the R package AHPtools publicly available on the Comprehensive R Archive Network (CRAN).