Task-Aware Morphology Optimization of Planar Manipulators via Reinforcement Learning

TL;DR

This paper demonstrates that reinforcement learning can effectively optimize the morphology of planar manipulators, recovering known optima and solving complex problems without analytical solutions, outperforming traditional search methods in efficiency.

Contribution

The study introduces RL-based morphology optimization for manipulators, showing its effectiveness in both analytical and non-analytical scenarios, with scalable performance compared to traditional methods.

Findings

RL recovers known optimal link configurations in simple cases.

RL outperforms grid and black-box methods in complex, high-dimensional tasks.

RL reliably converges in non-analytical morphology optimization problems.

Abstract

In this work, Yoshikawa's manipulability index is used to investigate reinforcement learning (RL) as a framework for morphology optimization in planar robotic manipulators. A 2R manipulator tracking a circular end-effector path is first examined because this case has a known analytical optimum: equal link lengths and the second joint orthogonal to the first. This serves as a validation step to test whether RL can rediscover the optimum using reward feedback alone, without access to the manipulability expression or the Jacobian. Three RL algorithms (SAC, DDPG, and PPO) are compared with grid search and black-box optimizers, with morphology represented by a single action parameter phi that maps to the link lengths. All methods converge to the analytical solution, showing that numerical recovery of the optimum is possible without supplying analytical structure. Most morphology design tasks have no closed-form solutions, and grid or heuristic search becomes expensive as dimensionality increases. RL is therefore explored as a scalable alternative. The formulation used for the circular path is extended to elliptical and rectangular paths by expanding the action space to the full morphology vector (L1, L2, theta2). In these non-analytical settings, RL continues to converge reliably, whereas grid and black-box methods require far larger evaluation budgets. These results indicate that RL is effective for both recovering known optima and solving morphology optimization problems without analytical solutions.