Efficient Learning in Chinese Checkers: Comparing Parameter Sharing in Multi-Agent Reinforcement Learning

TL;DR

This paper demonstrates that full parameter sharing in multi-agent reinforcement learning significantly improves performance in Chinese Checkers, a complex perfect-information game, by developing a new environment and applying advanced RL techniques.

Contribution

It introduces a faithful Chinese Checkers environment in PettingZoo and compares parameter sharing strategies in MARL, showing the superiority of full sharing in this context.

Findings

Full parameter sharing outperforms other architectures in Chinese Checkers.

Developed a new, faithful Chinese Checkers environment in PettingZoo.

Reduced action space complexity using branching actions.

Abstract

We show that multi-agent reinforcement learning (MARL) with full parameter sharing outperforms independent and partially shared architectures in the competitive perfect-information homogenous game of Chinese Checkers. To run our experiments, we develop a new MARL environment: variable-size, six-player Chinese Checkers. This custom environment was developed in PettingZoo and supports all traditional rules of the game including chaining jumps. This is, to the best of our knowledge, the first implementation of Chinese Checkers that remains faithful to the true game. Chinese Checkers is difficult to learn due to its large branching factor and potentially infinite horizons. We borrow the concept of branching actions (submoves) from complex action spaces in other RL domains, where a submove may not end a player's turn immediately. This drastically reduces the dimensionality of the action space. Our observation space is inspired by AlphaGo with many binary game boards stacked in a 3D array to encode information. The PettingZoo environment, training and evaluation logic, and analysis scripts can be found on \href{https://github.com/noahadhikari/pettingzoo-chinese-checkers}{Github}.