Design Process is a Reinforcement Learning Problem

TL;DR

This paper models the space layout planning process as a reinforcement learning problem, introduces an environment called RLDesigner for simulation, and demonstrates the use of PPO to optimize design layouts, bridging RL and architectural design.

Contribution

It formulates the design process as an RL problem, creates a customizable simulation environment, and applies PPO to layout planning, facilitating future research in RL-based design.

Findings

RL can effectively model design processes like SLP.

RLDesigner environment enables diverse scenario testing.

PPO shows promising results in layout optimization.

Abstract

While reinforcement learning has been used widely in research during the past few years, it found fewer real-world applications than supervised learning due to some weaknesses that the RL algorithms suffer from, such as performance degradation in transitioning from the simulator to the real world. Here, we argue the design process is a reinforcement learning problem and can potentially be a proper application for RL algorithms as it is an offline process and conventionally is done in CAD software - a sort of simulator. This creates opportunities for using RL methods and, at the same time, raises challenges. While the design processes are so diverse, here we focus on the space layout planning (SLP), frame it as an RL problem under the Markov Decision Process, and use PPO to address the layout design problem. To do so, we developed an environment named RLDesigner, to simulate the SLP. The RLDesigner is an OpenAI Gym compatible environment that can be easily customized to define a diverse range of design scenarios. We publicly share the environment to encourage both RL and architecture communities to use it for testing different RL algorithms or in their design practice. The codes are available in the following GitHub repository https://github.com/ RezaKakooee/rldesigner/tree/Second_Paper