# RoboCup 2D Soccer Simulation League: Evaluation Challenges

## TL;DR

This paper evaluates the 2016 RoboCup 2D Soccer Simulation League, establishing WE2015 as a benchmark, analyzing competition results, and proposing new evaluation formats to track progress and increase realism.

## Contribution

It provides a comprehensive analysis of 2016 competition results, confirms WE2015 as a stable benchmark, and introduces two innovative evaluation proposals for future challenges.

## Key findings

- WE2015 was the third in a round-robin tournament with top teams.
- WE2015 remains a stable benchmark for the league.
- Two new evaluation formats are proposed for future competitions.

## Abstract

We summarise the results of RoboCup 2D Soccer Simulation League in 2016 (Leipzig), including the main competition and the evaluation round. The evaluation round held in Leipzig confirmed the strength of RoboCup-2015 champion (WrightEagle, i.e. WE2015) in the League, with only eventual finalists of 2016 competition capable of defeating WE2015. An extended, post-Leipzig, round-robin tournament which included the top 8 teams of 2016, as well as WE2015, with over 1000 games played for each pair, placed WE2015 third behind the champion team (Gliders2016) and the runner-up (HELIOS2016). This establishes WE2015 as a stable benchmark for the 2D Simulation League. We then contrast two ranking methods and suggest two options for future evaluation challenges. The first one, "The Champions Simulation League", is proposed to include 6 previous champions, directly competing against each other in a round-robin tournament, with the view to systematically trace the advancements in the League. The second proposal, "The Global Challenge", is aimed to increase the realism of the environmental conditions during the simulated games, by simulating specific features of different participating countries.