Reinforcement Learning Algorithm for Mixed Mean Field Control Games

TL;DR

This paper introduces a reinforcement learning algorithm for solving mixed mean field control games, modeling competitive groups of agents with intra-group coordination and inter-group competition, with applications to financial trading scenarios.

Contribution

It proposes a novel three-timescale reinforcement learning algorithm tailored for mixed mean field control games with analytical benchmarks.

Findings

Algorithm successfully approximates Nash equilibria in benchmark problems.

The approach handles complex interactions between groups and individual agents.

Analytic solutions validate the effectiveness of the proposed method.

Abstract

We present a new combined \textit{mean field control game} (MFCG) problem which can be interpreted as a competitive game between collaborating groups and its solution as a Nash equilibrium between groups. Players coordinate their strategies within each group. An example is a modification of the classical trader's problem. Groups of traders maximize their wealth. They face cost for their transactions, for their own terminal positions, and for the average holding within their group. The asset price is impacted by the trades of all agents. We propose a three-timescale reinforcement learning algorithm to approximate the solution of such MFCG problems. We test the algorithm on benchmark linear-quadratic specifications for which we provide analytic solutions.