On the Exploration of LM-Based Soft Modular Robot Design

TL;DR

This paper explores leveraging large language models to assist in designing soft modular robots by translating user instructions into construction sequences, reducing trial-and-error, and enabling automatic evaluation of design quality.

Contribution

It introduces a novel approach using LLMs for robot design as a sequence generation task, incorporating simulation feedback and new evaluation metrics.

Findings

LLMs can effectively generate robot designs from natural language instructions.

Simulation-based feedback improves iterative design quality.

The approach successfully designs robots with locomotion and stair-descending capabilities.

Abstract

Recent large language models (LLMs) have demonstrated promising capabilities in modeling real-world knowledge and enhancing knowledge-based generation tasks. In this paper, we further explore the potential of using LLMs to aid in the design of soft modular robots, taking into account both user instructions and physical laws, to reduce the reliance on extensive trial-and-error experiments typically needed to achieve robot designs that meet specific structural or task requirements. Specifically, we formulate the robot design process as a sequence generation task and find that LLMs are able to capture key requirements expressed in natural language and reflect them in the construction sequences of robots. To simplify, rather than conducting real-world experiments to assess design quality, we utilize a simulation tool to provide feedback to the generative model, allowing for iterative improvements without requiring extensive human annotations. Furthermore, we introduce five evaluation metrics to assess the quality of robot designs from multiple angles including task completion and adherence to instructions, supporting an automatic evaluation process. Our model performs well in evaluations for designing soft modular robots with uni- and bi-directional locomotion and stair-descending capabilities, highlighting the potential of using natural language and LLMs for robot design. However, we also observe certain limitations that suggest areas for further improvement.