Interpretable classification of wiki-review streams

TL;DR

This paper presents an online, interpretable method for classifying wiki reviews to prevent vandalism, using real-time profiling and NLP features, achieving high accuracy and fairness through synthetic data balancing.

Contribution

It introduces a self-explainable, stream-based classification approach with a novel synthetic data algorithm for balanced, fair review classification in wiki environments.

Findings

Achieved near-90% accuracy, precision, recall, and F-measure.

Enabled real-time review classification with interpretability.

Improved fairness through synthetic data generation.

Abstract

Wiki articles are created and maintained by a crowd of editors, producing a continuous stream of reviews. Reviews can take the form of additions, reverts, or both. This crowdsourcing model is exposed to manipulation since neither reviews nor editors are automatically screened and purged. To protect articles against vandalism or damage, the stream of reviews can be mined to classify reviews and profile editors in real-time. The goal of this work is to anticipate and explain which reviews to revert. This way, editors are informed why their edits will be reverted. The proposed method employs stream-based processing, updating the profiling and classification models on each incoming event. The profiling uses side and content-based features employing Natural Language Processing, and editor profiles are incrementally updated based on their reviews. Since the proposed method relies on self-explainable classification algorithms, it is possible to understand why a review has been classified as a revert or a non-revert. In addition, this work contributes an algorithm for generating synthetic data for class balancing, making the final classification fairer. The proposed online method was tested with a real data set from Wikivoyage, which was balanced through the aforementioned synthetic data generation. The results attained near-90 % values for all evaluation metrics (accuracy, precision, recall, and F-measure).