Optimal Control of Agent-Based Dynamics under Deep Galerkin Feedback Laws

TL;DR

This paper addresses high-dimensional control problems using Deep Galerkin Methods, proposing a drift relaxation sampling approach that improves policy accuracy and reduces costs in mean-field models like opinion dynamics.

Contribution

It introduces a novel drift relaxation sampling technique to improve Deep Galerkin control methods for high-dimensional mean-field problems.

Findings

Significant cost reduction compared to manual control

Improved performance over Deep FBSDE in LQR problems

Effective in opinion dynamics models like Sznajd and Hegselmann-Krause

Abstract

Ever since the concepts of dynamic programming were introduced, one of the most difficult challenges has been to adequately address high-dimensional control problems. With growing dimensionality, the utilisation of Deep Neural Networks promises to circumvent the issue of an otherwise exponentially increasing complexity. The paper specifically investigates the sampling issues the Deep Galerkin Method is subjected to. It proposes a drift relaxation-based sampling approach to alleviate the symptoms of high-variance policy approximations. This is validated on mean-field control problems; namely, the variations of the opinion dynamics presented by the Sznajd and the Hegselmann-Krause model. The resulting policies induce a significant cost reduction over manually optimised control functions and show improvements on the Linear-Quadratic Regulator problem over the Deep FBSDE approach.