Towards Learning Rubik's Cube with N-tuple-based Reinforcement Learning

TL;DR

This paper presents a reinforcement learning approach using n-tuple representations and MCTS wrapping to learn and solve Rubik's cubes of sizes 2x2x2 and 3x3x3, achieving high success rates with manageable computational costs.

Contribution

It introduces a detailed method for learning Rubik's cube solving strategies using n-tuple reinforcement learning combined with MCTS, improving upon previous approaches.

Findings

Successfully solves 2x2x2 cubes completely.

Solves most 3x3x3 cubes with up to 15 twists.

Achieves lower computational costs than previous methods.

Abstract

This work describes in detail how to learn and solve the Rubik's cube game (or puzzle) in the General Board Game (GBG) learning and playing framework. We cover the cube sizes 2x2x2 and 3x3x3. We describe in detail the cube's state representation, how to transform it with twists, whole-cube rotations and color transformations and explain the use of symmetries in Rubik's cube. Next, we discuss different n-tuple representations for the cube, how we train the agents by reinforcement learning and how we improve the trained agents during evaluation by MCTS wrapping. We present results for agents that learn Rubik's cube from scratch, with and without MCTS wrapping, with and without symmetries and show that both, MCTS wrapping and symmetries, increase computational costs, but lead at the same time to much better results. We can solve the 2x2x2 cube completely, and the 3x3x3 cube in the majority of the cases for scrambled cubes up to p = 15 (QTM). We cannot yet reliably solve 3x3x3 cubes with more than 15 scrambling twists. Although our computational costs are higher with MCTS wrapping and with symmetries than without, they are still considerably lower than in the approaches of McAleer et al. (2018, 2019) and Agostinelli et al. (2019) who provide the best Rubik's cube learning agents so far.