Skill Issues: An Analysis of CS:GO Skill Rating Systems

TL;DR

This paper empirically evaluates popular skill rating systems in CS:GO, analyzing their real-world performance and data efficiency to improve fair matchmaking.

Contribution

It provides a comparative analysis of Elo, Glicko2, and TrueSkill using surrogate modelling on a large dataset, highlighting their practical strengths and weaknesses.

Findings

Glicko2 outperforms Elo and TrueSkill in data efficiency.

TrueSkill shows higher sensitivity to data variability.

All systems have trade-offs between accuracy and data requirements.

Abstract

The meteoric rise of online games has created a need for accurate skill rating systems for tracking improvement and fair matchmaking. Although many skill rating systems are deployed, with various theoretical foundations, less work has been done at analysing the real-world performance of these algorithms. In this paper, we perform an empirical analysis of Elo, Glicko2 and TrueSkill through the lens of surrogate modelling, where skill ratings influence future matchmaking with a configurable acquisition function. We look both at overall performance and data efficiency, and perform a sensitivity analysis based on a large dataset of Counter-Strike: Global Offensive matches.