A note on the empirical comparison of RBG and Ludii

TL;DR

This paper empirically compares the efficiency of three General Game Playing systems, demonstrating that RBG is generally faster than Ludii and GDL propnet, and critiques previous benchmarking methods for inaccuracies.

Contribution

It provides a comprehensive empirical comparison of RBG, Ludii, and GDL propnet, highlighting RBG's superior speed and addressing flaws in prior benchmarking studies.

Findings

RBG is about 37 times faster than Ludii for chess

Ludii is about 3 times slower than GDL propnet

Previous benchmarks had significant flaws leading to incorrect conclusions

Abstract

We present an experimental comparison of the efficiency of three General Game Playing systems in their current versions: Regular Boardgames (RBG 1.0), Ludii~0.3.0, and a Game Description Language (GDL) propnet. We show that in general, RBG is currently the fastest GGP system. For example, for chess, we demonstrate that RBG is about 37 times faster than Ludii, and Ludii is about 3 times slower than a GDL propnet. Referring to the recent comparison [An Empirical Evaluation of Two General Game Systems: Ludii and RBG, CoG 2019], we show evidences that the benchmark presented there contains a number of significant flaws that lead to wrong conclusions.