Reinforcement Learning for the Soccer Dribbling Task

TL;DR

This paper presents a reinforcement learning approach for a soccer dribbling task, where an agent learns to maintain possession against an adversary, demonstrating significant success after training.

Contribution

It introduces a novel RL framework with macro-actions and domain-specific state variables for soccer dribbling, achieving effective performance against a strong adversary.

Findings

Agent succeeds in 58% of trials after training

Uses Sarsa with CMAC for function approximation

Incorporates domain knowledge through macro-actions

Abstract

We propose a reinforcement learning solution to the \emph{soccer dribbling task}, a scenario in which a soccer agent has to go from the beginning to the end of a region keeping possession of the ball, as an adversary attempts to gain possession. While the adversary uses a stationary policy, the dribbler learns the best action to take at each decision point. After defining meaningful variables to represent the state space, and high-level macro-actions to incorporate domain knowledge, we describe our application of the reinforcement learning algorithm \emph{Sarsa} with CMAC for function approximation. Our experiments show that, after the training period, the dribbler is able to accomplish its task against a strong adversary around 58% of the time.