Off-Policy General Value Functions to Represent Dynamic Role Assignments in RoboCup 3D Soccer Simulation

TL;DR

This paper introduces off-policy gradient reinforcement learning methods to develop dynamic role assignment strategies in RoboCup 3D Soccer Simulation, enabling agents to learn competitive and adaptable policies in complex, adversarial environments.

Contribution

It applies off-policy gradient algorithms to learn real-time, dynamic role assignments, advancing knowledge representation and decision-making in multi-agent robotic soccer.

Findings

Agents learned competitive policies against top RoboCup teams.

The approach effectively models dynamic roles and group coordination.

Knowledge representations improved agent performance in complex scenarios.

Abstract

Collecting and maintaining accurate world knowledge in a dynamic, complex, adversarial, and stochastic environment such as the RoboCup 3D Soccer Simulation is a challenging task. Knowledge should be learned in real-time with time constraints. We use recently introduced Off-Policy Gradient Descent algorithms within Reinforcement Learning that illustrate learnable knowledge representations for dynamic role assignments. The results show that the agents have learned competitive policies against the top teams from the RoboCup 2012 competitions for three vs three, five vs five, and seven vs seven agents. We have explicitly used subsets of agents to identify the dynamics and the semantics for which the agents learn to maximize their performance measures, and to gather knowledge about different objectives, so that all agents participate effectively and efficiently within the group.