Double Deep Q-Learning in Opponent Modeling

TL;DR

This paper explores using Double Deep Q-Networks with a Mixture-of-Experts architecture for opponent modeling in multi-agent systems, demonstrating improved performance over standard DDQN in simulated environments.

Contribution

It introduces a novel combination of DDQN and Mixture-of-Experts for opponent modeling, enhancing strategy identification in multi-agent scenarios.

Findings

Mixture-of-Experts model outperforms DDQN in opponent strategy identification.

Opponent modeling improves agent performance in multi-agent environments.

The approach effectively captures diverse opponent strategies.

Abstract

Multi-agent systems in which secondary agents with conflicting agendas also alter their methods need opponent modeling. In this study, we simulate the main agent's and secondary agents' tactics using Double Deep Q-Networks (DDQN) with a prioritized experience replay mechanism. Then, under the opponent modeling setup, a Mixture-of-Experts architecture is used to identify various opponent strategy patterns. Finally, we analyze our models in two environments with several agents. The findings indicate that the Mixture-of-Experts model, which is based on opponent modeling, performs better than DDQN.