How does AI play football? An analysis of RL and real-world football strategies

TL;DR

This paper analyzes reinforcement learning football agents' strategies, comparing them with real players, and finds that increased competitiveness leads to more human-like play styles, with insights gained through social network analysis.

Contribution

It provides a novel comparison of RL football agents and real players, revealing how strategies evolve and become more human-like with training, using aggregated statistics and social network analysis.

Findings

Strong correlation between agent competitiveness and SNA metrics

RL agents' play styles become more similar to real players as they train

Insights into strategy development in simulated football environments

Abstract

Recent advances in reinforcement learning (RL) have made it possible to develop sophisticated agents that excel in a wide range of applications. Simulations using such agents can provide valuable information in scenarios that are difficult to scientifically experiment in the real world. In this paper, we examine the play-style characteristics of football RL agents and uncover how strategies may develop during training. The learnt strategies are then compared with those of real football players. We explore what can be learnt from the use of simulated environments by using aggregated statistics and social network analysis (SNA). As a result, we found that (1) there are strong correlations between the competitiveness of an agent and various SNA metrics and (2) aspects of the RL agents play style become similar to real world footballers as the agent becomes more competitive. We discuss further advances that may be necessary to improve our understanding necessary to fully utilise RL for the analysis of football.