Abnormal-aware Multi-person Evaluation System with Improved Fuzzy Weighting

TL;DR

This paper presents an improved multi-person evaluation system that detects anomalies and uses fuzzy weighting to enhance fairness and accuracy in subjective assessments.

Contribution

It introduces a two-stage screening with anomaly detection and fuzzy synthetic evaluation to improve impartiality and reliability in multi-person evaluations.

Findings

Effective filtering of abnormal data

More objective reviewer weighting

Clear and comprehensive ranking results

Abstract

There exists a phenomenon that subjectivity highly lies in the daily evaluation process. Our research primarily concentrates on a multi-person evaluation system with anomaly detection to minimize the possible inaccuracy that subjective assessment brings. We choose the two-stage screening method, which consists of rough screening and score-weighted Kendall-$\tau$ Distance to winnow out abnormal data, coupled with hypothesis testing to narrow global discrepancy. Then we use Fuzzy Synthetic Evaluation Method(FSE) to determine the significance of scores given by reviewers as well as their reliability, culminating in a more impartial weight for each reviewer in the final conclusion. The results demonstrate a clear and comprehensive ranking instead of unilateral scores, and we get to have an efficiency in filtering out abnormal data as well as a reasonably objective weight determination mechanism. We can sense that through our study, people will have a chance of modifying a multi-person evaluation system to attain both equity and a relatively superior competitive atmosphere.